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THE  EVOLUTIONARY  ROLE  OF  HUMANS 
IN  THE 

HUMAN-ROBOT  SYSTEM 

Thomas  M.  Granda,  Ph.D.  and  Mark  Kirkpatrick,  Ph.D. 
Carlow  Associates  Incorporated 
and 

Larry  A.  Peterson 

U.  S.  Army  Human  Engineering  Laboratory 


1-0  HUMAN-ROBOT  .SYSTEMS 

While  the  end  of  the  twentieth  century  will  be  remembered  as  the  time  of  the  early 
development  of  modern  robots,  the  twenty-first  century  may  very  weil  become  known  as  the 
century  of  the  robot.  Robot  technology  is  developing  at  a  rapid  pace.  So  much  so  that  many 
people  believe  that,  in  the  future,  robots  wili  not  be  considered  our  industrial  slaves,  but  our 
virtual  work  partners.  It  will,  however,  take  more  than  advanced  technology  to  turn  today's 
robot  into  a  true  work  partner.  To  bring  about  this  transition,  human  factors  will  have  to  make 
a  major  contribution  toward  defining  the  human's  changing  role  in  relation  to  changing  robot 
technology. 

The  technological  evolution  of  robots  has,  and  wiil  continue  to  have,  important  implications 
for  the  division  of  labor  between  humans  and  robots  (1,  2).  Today's  modern  robots  are 
primarily  used  in  well  defined  and  repetitive  tasks  in  the  context  of  manufacturing.  This  use  of 
robots  still  coincides  with  the  origin  of  the  word  "robot.’’  It  is  derived  from  the  Czechoslovakian 
wora  "lobota"  which  means  compulsory  labor  and  drudgery  (3). 

Tomorrow's  robots  will  be  used  for  far  more  than  physical  labor.  Robots  will  become  much 
more  capable  of  emulating  human  sensing  and  cognitive  processes.  As  this  occurs,  there  will 
be  a  shift  from  viewing  robots  as  independent  machines  to  cognitive  components,  along  with 
humans,  of  a  Human-Robot  System  (HRS).  The  robotic  metamorphosis  will  concurrently  cause 
the  role  of  the  operator  in  the  HRS  to  undergo  a  major  transformation.  This  change  will  not  only 
redefine  the  operators  work  role  in  relation  to  robots,  but  may  well  redefine  our  point  of  view 
toward  advanced  technology  and  science. 

The  purpose  of  this  paper  is  to  identify  some  general  HRS  functions  and  to  estimate  the 
criticality  of  those  functions,  in  terms  of  system  effectiveness,  for  first  line  operations 
personnel  in  the  system. 

2.0  STAGES  OP  HRS  CAPABILITY 

The  development  of  HRS  functional  capabilities  can  be  categorized  into  several  stages  as 
follows: 


•  Bounded  autonomy 

•  Teleoperation 

•  Supervised  autonomy 

•  Adaptive  autonomy 

•  Virtual  symbiosis 

Some  characteristics  of  these  stages  are  summarized  in  Table  1.  The  stages  are  not  mutually 
exclusive  and  often  overlap  with  characteristics  of  one  stage  blurring  into  the  next.  With  that  in 
mind,  the  next  few  paragraphs  briefly  describe  each  stage. 
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Table  1.  Capability  Stages  in  the  Development 
of  Human-Robot  Systems 


Stage 

Characteristics 

Example 

Information  Processing 

Bounded 

Autonomy 

•  Limited  task  domain 

•  Specialized  tasks 

•  Inflexible  response 

•  No  task  uncertainty 

•  Industrial  robots 
Production  tasks 
Welding 
Painting 

•  Fixed  prior  to  task  execution  via: 

Teach  pendant 

Detailed  manual  inputs 

Fixed  repertoire  of  movements/actions 

•  Manipulator  joint  positicn/rate  commands 

Teleoperation 

•  Generalized 
platform/tools 

•  Mobility 

•  Flexibility 

•  Operator  in  loop  full 
time 

•  Shuttle  Orbiter 
Remote  Manipulator 
System 

•  Extension  of  human  capabilities 

•  Unprocessed  sensor  feedback  to  operator 

•  Continuous  loop  closure  by  operator 

•  Manipulator  joint  position/rate  commands 

•  Translation  to  object/work  space 
coordinate  frame,  movement  constraints 

Supervised 

Autonomy 

•  World  modeling 

•  Variable  degree  of 
world  structure 

•  Variable  span  of 
autonomous  control 

•  Task  orientation 

•  Hubble  Space 
Telescope 

•  JPL  CARD  -  lunar/ 
planetary  rover 

•  Space  Station 
payload  servicing 

•  Generalized  tasks  and  parameters  defined 
by  operator 

•  Processed  sensor  feedback  &  local  loops 

•  Operator  intervention  at  interval  Dt 

•  Task/argument  command  language 

•  World  model  controlled  by  operator 

Adaptive 

Autonomy 

•  Complex  tasks  and 
task  sequences 

•  Machine  pattern 
recognition 

•  Advanced  world 
modeling 

•  Task  sequence/ 
mission  orientation 

•  Pilot's  Associate 

•  Decision  making  shared  by  operator  & 
machine 

•  Operator  intervention  by  exception 

•  Multi-sensor  fusion 

•  Machine  world  model  updating  & 
management 

•  Machine  extrapolation  of  world  knowledge 

•  Task  sequence/argument  dialog  language 

Virtual 

Symbiosis 

•  Machine  functions 
as  "co-worker" 

•  Goal  orientation 

•  Autonomous  task 
implementation  and 
goal-directed 
learning 

•  Periodic  validation 
by  operator 

0 

■ 

Human  manual  labor  serves  as  a  baseline  for  comparison.  The  typical  division  of  labor 
among  workers  in  this  baseline  falls  into  such  duties  as  direct  labor,  foreman,  supervisor,  and 
manager.  These  divisions  somewhat  parallel  the  evolution  of  the  human  role  in  HRSs. 
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2.1  B-oundeti  Autonomy 

Robotic  devices  in  the  bounded  autonomy  stage  function  with  little  human  intervention 
having  been  pre-programmed  to  produce  very  specific  movement/action  sequences.  Such 
devices  are  frequently  used  in  repetitive  and  welt  defined  tasks  such  as  those  on  production 
lines.  Such  robots  possess  zero  adaptability  and  required  reprogramming  if  the  task  is  changed. 
Once  programmed  by  means  of  teach  pendants  or  off-line  programming,  they  execute  component 
actions  with  high  reliability. 

2.2  Teleoperation 

In  the  teleoperation  stage,  which  includes  subsystems  such  as  mobility  platforms, 
manipulators,  sensors,  and  end-effectors,  the  human  is  an  integral  and  full  time  component  in 
the  control  loop.  The  control  loop  is  closed  only  by  the  human  operator  who  defines  the  task  to 
be  performed  and  operates  the  controls  to  perform  that  task.  The  operator  uses  feedback 
(usually  visual,  but  sometimes  including  force  or  audio  feedback)  from  the  remote  site  but  the 
significant  point  is  that  the  operator  is  responsible  for  all  of  the  active  processing  of 
information  aside  from  some  low-ievel  operations  such  as  coordinate  transformations.  The 
robot  responds  to  the  commands  of  the  human  and  not  directly  to  variables  in  the  enviromsai-.t. 
Typically,  in  this  stage,  the  remote  robotic  device  is  often  used  to  avoid  danger  to  the  human 
operator  or  to  operate  at  distant  worksites. 

2.3  Supervised  Autonomy 

The  supervisory  stage  refers  to  the  HRS  level  in  which  the  robot  can  handle  certain  tasks 
or  subtasks  autonomously  allowing  the  human  to  perform  other  tasks  or  control  other  robots. 
Typically,  in  this  stage,  the  human  is  responsible  for  planning  what  to  do  and  how  to  do  it, 
teaching  the  computer,  monitoring  robot  activities,  intervening  in  robot  tasks  when  necessary, 
and  learning  how  to  perform  the  totality  of  tasks  better  in  the  future  (4,5).  It  is  in  this  stage 
that  true  human  and  robot  interactivity  begins.  This  new  level  of  interactivity  warrants  special 
attention  to  the  safety  of  humans  (6).  In  the  supervised  autonomy  stage,  the  human  operator 
defines  a  specific  task  (such  as  road  following)  with  parameters  (such  as  speed)  which  the 
remote  system  performs  to  completion  with  occasional  operator  intervention  after  the  initial 
task  assignment.  The  robotic  system  will  typically  incorporate  some  types  of  onboard 
intelligence  (e.g.,  a  world  model)  as  well  as  primitive  sensing  and  processing  capabilities  (e.g., 
obstacle  detection  sensors  and  avoidance  algorithms).  The  system  would  still  require  human 
intervention  in  the  cases  where  its  capabilities  are  inadequate  for  an  unexpected  event  or 
circumstance. 

2.4  Adaptive,  Autonomy. 

In  the  adaptive  autonomy  stage  the  robot  can  perform  many  tasks  independently  of  the 
human  operator.  The  human  will  be  needed  less  in  the  operational  control  loop  but  will  be 
retained  in  the  system  loop  (7).  It  is  in  this  stage  that  the  human  functions  as  an  information 
manager;  one  who  organizes  the  structure  of  expert  systems  and  data  bases  including  world 
models,  and  decides  what  knowledge  will  be  imparted  to  the  data  base. 

Adaptive  autonomy  is  characterized  by  the  robot's  ability  to  learn  from  experience.  That 
is,  the  robot  incorporates  environmental  and  procedural  knowledge  learned  in  one  task  that  may 
be  used  in  the  performance  of  future  tasks.  This  presupposes  machine  capabilities  in  world 
model  updating/maintenance  and  automatic  programming.  Human  intervention  will  be  by 
exception  where  incorrect  machine  proposals  must  be  over  ridden.  Safety  monitoring  will 
become  a  critical  task  for  humans  in  this  stage  because  the  human  must  be  sure  that  what  is 
learned  is  not  detrimental  to  higher  level  goals. 
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in  the  final  stage  there  is  virtual  symbiosis  between  the  human  and  the  robot.  The  human 
and  the  robot  will  be  able  to  exchange  data,  information,  and  recommendations  at  a  fairly  high 
cognitive  level  using  languages  which  can  express  machine  proposed  actions  and  operator 
consent  or  rejection.  The  shared  knowledge  will  extend  to  matters  of  planning,  maintenance,  and 
programming  but  the  ultimate  responsibility  for  these  functions  reside  with  the  human 
operator. 


An  attempt  was  made  to  define  changes  in  the  human  role  in  HRSs  under  the  various 
capability  stages  by  estimating  the  criticality  for  acceptable  system  performance  of  certain 
generic  classes  of  operator  functions  for  each  of  the  stages. 


Seven  generalized  functions  of  human  operators  in  an  HRS  were  defined  as  follows: 


•  Labor  intensive  work 

•  Task  planning 

•  Mission  planning 

•  Monitoring  and  intervention 

•  Information  management 

»  Interactivity 

These  functions  are  seen  as  being  performed  by  the  human  operator  who  most  directly 
interacts  with  the  robot  during  mission  operations.  They  do  not  address  activities  performed  by 
supervisory  personnel, nor  do  they  address  HRS  subsystem  activities  such  as  monitoring,  fault 
detection,  maintenance  and  repair. 

3.1.1  Labor  intensive  work.  This  factor  refers  to  tabor  requirements  which  involve 
physical  exertion. 

3.1.2  Task  planning.  Planning  at  the  task  level  entails  development  of  plans  which 
will  accomplish  task  goals.  The  term  task  refers  to  short-term  low  level  activities  such  as 
moving  an  object  to  a  specified  position. 

3.1.3  Mission  planning.  Planning  at  the  mission  level  entails  development  of  plans 
which  will  accomplish  mission  goals.  Such  plans  address  longer  term,  high  level  objectives. 
Mission  planning  may  also  involve  decomposition  of  the  mission  problem  into  parts  which 
provide  subgoals  at  which  task  planning  is  directed. 

3.1.4  Monitoring  and  intervention.  This  function  involves  real-time,  periodic  or 
time  sampled  evaluation  of  processes,  and  current  states  of  objects  or  variables  in  the 
environment  to  verify  that  plans  have  been  completed  and  goals  met  or  that  plans  are  being 
executed  satisfactorily.  If  the  process  is  not  acceptable,  then  some  form  of  operator  intervention 
will  be  required. 

3.1.5  Information  management.  Information  management  includes  processing  of 
information  to  support  decision  making,  and  selection  of  information  elements  for  attention, 
reliability  evaluation,  storage  and  dissemination. 

3.1 .6  Interactivity.  This  function  refers  to  mutually  dependent  actions  and 
reactions  on  the  parts  of  both  the  human  operator  and  the  machine.  It  is  not  the  same  as 
workload  or  rate  of  control  actuation  which  may  be  extremely  high  in  a  teleoperated  HRS. 

Rather,  interactivity  involves  rapid  changes  in  allocation  of  functions  between  human  and 
machine.  Essentially,  interactivity  refers  to  rapid  and  frequent  changes  in  "whose  court  the  bail 
is  in"  with  respect  to  each  of  the  currently  ongoing  functions. 
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3-2  Estimation  of  Function  Criticality 

The  authors  attempted  to  estimate  ihe  criticality  of  each  of  the  above  generic  human 
functions  for  a  hypothetical  HRS  at  each  of  the  capability  stages  discussed  previously. 

Criticality,  as  used  here,  refers  to  the  importance  of  connect  and  tims!v  performance  of  a 
particular  function  for  the  HRS  as  well  as  to  the  necessity  for  the  appropriate  supporting 
technology.  The  ratings  developed  do  not  refer  to  the  toad  or  time  spent  on  the  function. 

Criticality  estimates  utilized  a  scale  from  zero  to  ten  with  zero  meaning  that  the  function  has  no 
importance  and  ten  meaning  that  the  hypothetical  system  will  not  be  feasible  if  the  function 
cannot  be  performed.  Function  loadings  in  terms  of  percent  of  effort  devoted  to  the  function 
would  be  too  system  and/or  mission  specific  to  be  treated  in  a  general  fashion. 

3.3  Results  of  Evaluation 

The  results  of  the  criticality  estimation  process  are  shown  in  Figure  1 .  For  comparison 
purposes,  the  manual  work  mode  without  robotic  involvement  is  included  as  a  baseline.  The 
Figure  1  data  represent  a  consensus  on  the  part  of  the  authors  with  regard  to  the  criticaiity  of 
the  various  generic  operator  functions  throughout  the  capability  stages.  The  criticality 
estimates  are  believed  to  provide  an  approximate  profile  of  the  changing  role  of  the  human 
operator  in  HRSs  as  development  proceeds  through  the  several  stages.  The  remainder  of  this 
section  will  evaluate  the  human  role  profile  in  the  HRS  criticality  graphic.  The  following 
section  will  discuss  some  broad  implications  of  the  changing  human  roies  in  the  HRS  tor  human 
factors. 
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Figure  1,  Criticality  Ratings  by  HRS  Stages  and  Generic  Operator  Functions 

3.3.1  Interactivity.  With  increasing  complexity  there  is  a  shift  from  user-display 
interface,  which  monitors  unprocessed  data,  to  a  user-system  interface  which  presents  the 
operator  with  summarized  data,  probabilistic  estimates,  and  extrapolations  of  system 
operational  events.  Human  factors  will  play  a  major  role  in  structuring  and  presenting  tile 
information  to  the  operator. 


AC/243-TP/3 


Cl. 6 


3.3.2  Information  management.  With  increasing  system  complexity,  the  operator 
plays  an  increasingly  important  role  in  managing  information  that  is  collected  by  the  HRS. 
Human  factors  must  help  to  design  information  management  aides  that  will  help  the  operator  to 
make  information  decisions  (e.g.,  what  information  to  keep  or  purge,  what  level  of  access 
the  robot  should  have  to  the  new  information,  what  should  be  shared  with  other  HRSs,  and  what 
might  be  the  impact  of  system  peformance  and  safety  of  information). 

3.3.3  Monitoring  and  intervention.  With  increased  system  complexity  the  criticality 
of  human  situational  awareness  during  the  monitoring  process  increases.  This  implies  a 
definite  need  for  a  cognitive  system  structure  representation.  With  increasing  complexity  the 
ability  of  the  operator  to  understand  what  the  system  is  doing,  and  why,  will  be  limited.  Human 
factors  will  have  to  explore  methods  of  continually  representing  system  functions  to  the 
operator  in  order  to  increase  the  likelihood  that  the  operator  will  be  able  to  play  a  role  not  only 
in  dealing  with  system  problems  as  they  occur,  but  perhaps  more  importantly,  in  anticipating 
system  problems. 

An  especially  difficult  problem,  which  already  occurs  in  nuclear  reactor  control  rooms  and 
advanced  technology  aircraft,  involves  the  idea  of  an  "interval  of  uncertainty."  An  "interval  of 
uncertainty"  can  be  defined  as  a  period  of  system  operations  which  is  characterized  by  rapidly 
decaying  system  integrity,  accompanied  by  little  or  no  information  as  to  the  cause(s).  With 
increasing  system  complexity  the  probability  of  encountering  an  "interval  of  uncertainty" 
increases.  Human  factors  will  be  instrumental  in  devising  methods  and  procedures  for 
acquiring  and  processing  information  during  this  time. 

3.3.4  Mission  planning.  With  increasing  advanced  robot  technology,  the  robot  will 
perform  much  of  the  task  planning  and  execution.  As  a  necessary  step  in  integrating  the  human 
and  robot  elements  of  the  system,  the  human  will  become  more  involved  in  concatenating  tasks 
to  the  level  of  mission  requirements.  Human  factors  will  deal  with  the  cognitive  aspect  of  aiding 
the  operator  to  produce  feasible  and  effective  mission  plans. 

3.4  Implications  of  Results 

3.4.1  Languages.  Planning/operations  languages  will  be  required  to  support  the 
mission  and  task  planning  functions.  To  a  large  extent,  robotic  programming  languages  support 
off-line  programming  of  assembly  operations  and  are  not  particularly  suited  for  real-time, 
on-line  control  of  robots.  The  Space  Station  User  Interface  Language  (SSUIL)  is  currently  under 
development  to  support  Space  Station  science  users  in  planning  the  operation  of  experiment 
payloads.  It  is  object  and  action  oriented  and  contains  provision  for  a  direct  manipulation  user 
interface.  These  language  features  will  go  far  toward  providing  the  task  and  mission  planning 
capabilities  required  for  the  supervised  autonomy  and  higher  stages.  It  is  considered  that  an 
essential  development  feature  of  the  necessary  robot  control  language  will  be  cognitive  analysis 
of  user  tasks  and  knowledge  structures. 

3.4.2  Operator-computer  interface.  The  above  discussion  of  roboiic  system 
planning/operations  languages  does  not  mean  that  a  strictly  command  language  dialog  is  essential 
or  even  contemplated.  The  user  interface,  whereby  commands  in  the  language  are  composed, 
might  support  writing  commands  directly  but  other  dialog  types  including  direct  manipulation 
may  well  be  more  effective  for  on-line  operator  control. 

3.4.3  Interaction  with  a  world  model.  Task  and  mission  planning  automation  in  the 
supervised  autonomy  and  higher  stages  will  require  a  world  model.  Where  the  working 
environment  is  man-made,  it  is  readily  defined  for  world  modeling  purposes  as  in  the  case  of 
payload  setvicing  robots  planned  for  the  Space  Station  (8).  Where  natural  objects  and  terrain 
are  concerned,  advances  in  machine  pattern  recognition  will  eventually  support  machine 
updating  of  a  world  model.  In  the  near  term,  however,  a  human  operator  will  have  to  input 
corrections  to  a  world  model.  Three  dimensional  imagery  and  fast  coordinate  transformations  to 
support  multiple  views  are  likely  to  be  essential  to  the  necessary  interfaces. 
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3.4.4  Monitoring  and  intervention.  As  automated  HRS  capabilities  increase,  the 
frequency  of  operator  intervention  will  be  decreased.  If  this  frequency  becomes  low  enough,  long 
periods  of  uneventful  monitoring  could  result,  with  associated  vigilance  decrements  etc.  This 
could  become  a  human  factors  concern.  Provision  will  be  required  for  “gr?.ce1u!  stage 
regression"  in  certain  cases  of  operator  intervention.  If  the  automated  elements  of  the  system 
are  incapable  of  producing  a  certain  evolution,  the  operator  may  have  to  select  a  lower  stage 
(such  as  teleoperation).  This  point  has  been  well  made  by  Aibus,  McCain  and  lumia  (8). 
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1.  INTRODUCTION 

1 .  It  is  reasonable  to  project  that  future  military  operations  will  take  place  in  increasingly 
hostile  environments.  The  military's  effectiveness  in  controlling  people  and  equipment  will 
depend  on  its  ability  to  operate  in  such  environments.  Advances  in  mobility,  firepower  and 
communication  will  permit  greater  dispersion  of  soldiers  and  thereby  reduce  somewhat  the 
effects  of  hazards.  Even  greater  protection  for  military  personnel  will  be  provided  by  efficient 
deployment  of  teleoperated  and  telemanaged  robots,  however. 

2.  For  the  foreseeable  future,  these  remotely  controlled  systems  will  make  use  of 
human  facilities  to  achieve  effectiveness  in  control  functions,  especially  in  such  predominantly 
“unstructured”  environments  as  the  battlefield,  where  completely  autonomous  robotics  is  still 
far  from  being  realisable.  It  is  therefore  essential  to  develop  optimal  capabilities  for  these 
remotely  operated  systems,  to  suit  the  different  levels  of  structuring  of  their  operational 
environments. 

3.  The  effectiveness  of  such  systems  will  be  determined  by  the  quality  of  the 
interactions  between  the  operator  and  the  remote  robot.  The  quality  of  the  robot-to-operator 
component  of  the  interaction  will  be  particularly  important.  The  robot's  sensing  capabilities  and 
the  means  of  delivering  the  information  to  the  operator  will  determine  the  quality  of  his 
perceptions  on  the  status  of  the  robot  and  its  environment,  and  thereby  affect  the  operator's 
decisions  and  resulting  actions.  A  well  developed  coupling  between  multiple  remote  sensors  on 
the  robot  and  the  operator  will  provide  a  capability  approaching  "telepresence".  As  for  the 
operator-to-robot  component,  the  required  telepresence  capabilities  must  be  such  that  the 
operator-robot  combination  will  be  able  to  effectively  carry  out  mobility  and  manipulation  tasks, 
as  well  as  use  its  weapons. 

4.  One  necessary  condition  for  effective  teleoperation  of  weapons  is  that  dimensional 
information  about  the  target,  and  its  distance,  elevation,  and  bearing  from  the  robot  to  the  target, 
be  accurately  estimated.  A  similar  criterion  applies  with  respect  to  obstacle  avoidance  in 


1  The  work  presented  here  was  carried  out  under  contract  W771 1-7-7009/01-SE  with  Supply  and  Services  Canada 
for  the  Defence  and  Civil  Institute  of  Environmental  Medicine,  North  York,  Ontario,  Canada 

2  Email:  jul@dretor.dciem.dnd.ca 

3  Email:  milgram@gpu.utcs.utoronto.ca 

4  Email:  drascic@phoenuuose.utoronto.ca 


AC/243-TP/3 


C2.2 


mobility  operations,  and  robotic  or  teleoperated  manipulation  operations.  This  requirement 
becomes  particularly  pronounced  when  the  speed  or  rate  of  task  execution  is  a  critical  factor. 
Consequently,  to  permit  successful  and  effective  teleoperations,  it  is  important  to  provide  the 
operator  with  accurate  spatial  information  about  the  remote  site. 

5 .  The  principal  feedback  link  with  the  telerobot  is  the  video  display.  Conventional 
video  sensors  alone,  however,  do  not  provide  the  required  spatial  information  directly.  In  this 
paper  we  discuss  an  ongoing  project  to  develop  a  hybrid  display  system  which  combines  a 
stereoscopic  video  system  with  a  stereoscopic  graphics  system,  thereby  providing  operators 
with  greatly  enhanced  information,  both  relative  and  absolute,  about  the  remote  worksite.  The 
paper  first  presents  a  brief  review  of  the  benefits  of  stereoscopic  video  in  telerobotics.  We  then 
point  out  some  of  the  operational  issues  associated  with  manual  teleoperation  which  remain, 
even  after  stereo  video  is  introduced.  The  configuration  of  the  new  display  system  developed 
to  address  these  operational  problems  is  reviewed,  along  with  some  current  and  future 
fundamental  capabilities.  Finally,  the  project  status  is  summarised. 


2.  OPERATIONAL  BENEFITS  OF  STEREOSCOPIC  VIDEO 

6.  It  is  now  widely  accepted  that,  for  a  large  number  of  tasks,  provision  of  a  basic 
stereoscopic  video  (SV)  capability  can  enhance  teleoperator  performance  significantly  (Menu; 
1988).  An  overview  of  visual  performance  research  with  SV  systems  can  be  found  in  Spain 
(1984).  Some  of  the  benefits  of  SV  systems  which  pertain  specifically  to  land-based  military 
operations  are  summarised  below. 

2.1  Relative  Judgement  of  Object  Location 

7 .  In  general,  the  great  majority  of  electronic  display  media  are  monoscopic,  that  is,  the 
images  which  are  perceived  are  equivalent  to  those  which  would  be  viewed  in  nature  with  one 
eye  only.  Such  display  systems  do  contain  numerous  depth  cues,  including:  relative  size  of 
objects  viewed,  geometric  perspective,  occlusion,  lighting,  shading,  relative  motion,  visual 
flow  patterns,  etc.  It  is  nevertheless  well  known  that,  in  supplying  extra  stereoscopic  cues 
(which  current  research  is  indicating  is  additive  with  respect  to  the  other  (monoscopic)  cues 
(Sollenberger  et  al,  1991;  Dosher  et  al,  1986)),  the  ability  to  estimate  where  objects  are  in  the 
visual  field,  relative  to  each  other,  is  greatly  improved.  Operations  which  require  that  this  be 
done  accurately  include  most  manipulative  activities,  reconnaissance,  aiming  of  weaponry,  etc. 
In  general,  accurate  perception  of  obstacle  locations  and  dimensions  permits  efficient  execution 
of  critical  manoeuvres  and  use  of  clearances  to  effect  passage  of  a  robot  or  its  manipulator  arm 
in  a  safe,  collision-free  manner. 

2.2  Enhanced  Image  Interpretation 

8.  In  addition  to  providing  a  significant  improvement  in  seeing  where  things  are, 
stereoscopic  viewing  can  also  enhance  the  ability  to  perceive  what  things  are,  especially  in 
unfamiliar  or  complex  visual  environments.  This  is  especially  important"  for  reconnaissance, 
camouflage  detection,  logistics  planning,  ordnance  management,  etc.  For  example,  a 
camouflaged  vehicle  stationed  in  front  of  a  forest  may  be  completely  inconspicuous  using  a 
monoscopic  display,  yet  may  stand  out  dramatically  on  a  stereoscopic  display. 
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2.3  Enhanced  Task  Performance 

9.  Some  of  the  remote  handling  tasks  for  which  stereoscopic  viewing  is  particularly 
helpful  are  those  which  involve  ballistic  movement,  recognition  of  unfamiliar  scenes,  analysis 
of  three  dimensionally  complex  scenes,  and  the  accurate  placement  of  manipulators  or  tools 
within  such  scenes  (Dumbreck  et  al,  1987).  Results  of  other  studies  suggest  that  stereoscopic 
viewing  is  critical  when  tasks  are  unpredictable  or  constantly  changing,  when  visibility 
conditions  are  poor,  or  when  precise  positioning  in  depth  is  critical  for  success. 

2.4  Enhanced  Slope  and  "Negative  Obstacle"  Perception 

10.  Experiments  in  off-road  driving  have  shown  that  it  is  very  difficult  to  perceive  terrain 
slopes  and  to  detect  "negative  obstacles",  that  is,  sudden  dips  or  drop-offs,  with  monocular 
viewing.  Stereoscopic  viewing  can  enhance  perception  considerably  under  such  situations. 
This  is  especially  important  in  batdefield  telerobotics,  where  it  is  quite  common  for  teleoperated 
vehicles  to  roll  into  hidden  gullies  and  topple  over  on  inclined  slopes. 

2.5  Enhanced  Training  Regimes 

11.  The  eye-body  relationship  has  a  strong  bearing  on  human  spatial  perception,  as  a 
consequence  of  life-long  experience  and  learning.  In  remote  viewing,  therefore,  the  operator's 
own  frame  of  reference  tends  to  dominate  over  the  camera-robot  frame  of  reference.  For  those 
systems  in  which  the  camera  is  mounted  on  the  robot  platform  to  retain  a  constant  point  of  view 
with  respect  to  the  end  effector,  perception  is  also  affected  by  frequent  movement  of  the  camera 
system  during  operations  of  the  robot  arm.  The  consequences  of  such  factors  do  not  manifest 
themselves  in  slow,  simple  operations  relative  to  a  distant  horizon.  However,  they  do  become 
particularly  evident  in  busy  environments  involving  rapid  critical  operations.  Any  inadequacies 
in  the  operator’s  perception  of  spatial  information  can  therefore  severely  limit  the  efficiency  of 
some  teleoperated  activities.  At  the  same  time,  a  trial-and-error  approach  to  compensate  for 
these  inadequacies  will  not  be  acceptable  under  operational  settings. 

12.  Most  research  with  stereoscopic  displays  has  concentrated  on  showing  either 
improved  depth  resolution  or  improved  task  performance  relative  to  monoscopic  viewing.  In 
some  situations,  operators  with  two  orthogonal  single  camera  views  can  be  as  effective  as  those 
with  one  stereoscopic  view,  with  sufficient  training.  For  highly  repetitive  tasks,  furthermore,  it 
is  possible  for  skilled  operators  to  perform  well-rehearsed  teleoperation  tasks  as  effectively 
using  a  single  monoscopic  view.  Most  teleoperation  tasks  are  rarely  repetitive,  however,  and 
especially  not  battlefield  teleoperauons. 

13.  As  with  other  operational  systems,  one  of  the  most  important  aspects  of  managing  a 
telerobot  system  is  the  investment  needed  to  train  operators  to  use  it.  Operators  of  current 
explosive  ordnance  disposal  robots  are  subjected  to  extensive  and  costly  practical  training 
toward  proficiency  in  teleoperation.  Periodic  refresh  training  is  also  common,  to  maintain  their 
operational  readiness.  Preliminary  research  has  shown  that  transfer  effects  arising  from  early 
exposure  to  manual  teleoperation  using  stereoscopic  video  are  positive  (Drascic  et  al,  1989), 
implying  that  the  use  of  stereoscopic  viewing  may  significantly  reduce  operational  training 
requirements. 
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3.  OPERATIONAL.ISSUES,  ASSOCIATED  WITH  STEREOSCOPIC  VIDEO 

14.  In  the  preceding  section,  several  of  the  advantages  of  stereoscopic  viewing  systems 
have  been  reviewed.  It  is  our  contention,  however,  that  further  operational  enhancements  can 
be  achieved  by  extending  the  SV  capability  to  encompass  stereoscopic  computer  graphics  and 
machine  vision  technologies.  The  issues  addressed  include  the  following: 

3.1  Estimation  of  Absolute  Distances  /  Locations 

15.  In  general,  when  using  monoscopic  video  systems  it  is  very  difficult  and  it  requires  a 
great  deal  of  training  and  practice  to  estimate  object  sizes  and  distances  accurately.  In  unf  amiliar 
or  obscured  environments,  this  can  be  almost  impossible.  Users  of  stereoscopic  video 
systems,  on  the  other  hand,  benefit  from  a  greatly  enhanced  ability  to  discriminate  relative 
distances,  depths  and  object  sizes;  that  is,  the  relative  locations  of  objects  in  space  are 
immediately  obvious  to  most  observers.  Unless  observers  are  very  well  trained,  and  very 
familiar  with  the  objects  in  the  remote  location,  however,  they  are  still  somewhat  limited  in  their 
ability  to  make  absolute  judgements  of  distance  and  depth.  In  other  words,  whereas  it  might  be 
easy  for  an  observer  to  determine  that  object  A  is  farther  away  from  the  cameras  than  object  B, 
for  example,  it  is  generally  more  difficult  to  estimate  how  far  away  the  two  objects  are  from  the 
cameras,  or  from  each  other.  The  ability  to  make  such  estimates  can  be  of  importance  for  tasks 
involving  remote  surveillance,  assembly,  telerobotic  path  planning,  obstacle  avoidance,  weapon 
deployment,  etc. 

3.2  Specification  and  Adjustment  of  Stereoscopic  Camera  Parameters 

1 6.  The  essence  of  a  stereoscopic  display  system  is  that  images  from  equivalent  left  and 
right  eye  views  are  displaced  with  respect  to  each  other  when  presented  on  a  monitor.  The 
viewer-monitor  geometry  determines  to  what  extent  the  resulting  binocular  disparity  is 
interpreted  as  depth.  If  lines  were  drawn  along  the  optical  axes  of  the  viewer’s  eyes  to 
corresponding  points  of  left  and  right  images  on  the  monitor,  and  then  extended  until  they 
intersect,  this  would  determine  the  effective  convergence  point  of  the  observer's  eyes. 

17.  Relating  this  to  natural  vision,  human  eyes  can  typically  converge  to  a  point  as  close 
as  15  cm  and  as  far  away  as  infinity.  That  is,  with  natural  vision  one  sees  what  is  actually 
present  at  distances  as  close  as  15  cm  from  one’s  face.  When  using  stereoscopic  video, 
however,  what  the  observer  sees  at  that  apparent  distance,  called  the  "near  point",  is  a  function 
of  the  separation,  convergence  angle  and  focal  length  of  the  cameras,  as  well  as  the  size  of  the 
monitor  and  the  observer's  position  relative  to  it.  These  same  parameters  also  determine  what 
appears  at  "infinity"  for  the  observer,  known  as  the  "far  point".  If  the  near  point,  and  the  far 
point  are  close  together,  observers  are  limited  in  what  they  can  comfortably  see;  however,  what 
they  can  see,  they  see  with  much  higher  depth  resolution  than  would  typically  be  possible  with 
natural  vision  (despite  the  degradation  of  image  quality  due  to  the  video  medium).  This  is 
similar  to  the  function  of  binoculars,  which  move  the  near  point  further  away,  leaving  the  far 
point  at  infinity,  and  so  effectively  increase  the  depth  resolution  within  that  range. 

1 8.  In  terms  of  stereoscopic  video  systems,  for  object  distances  which  appear  very  close 
relative  to  the  screen  depth,  the  viewer’s  eyes  arc  forced  to  converge  excessively,  resulting  in 
possible  eyestrain.  Conversely,  for  very  far  distances  relative  to  the  screen  depth,  the  eyes 
might  even  be  forced  to  diverge.  In  addition,  problems  may  develop  because  the  observer’s 
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eyes  tend  to  focus  at  the  depth  of  the  monitor  screen,  while  converging  at  a  virtual  depth  point 
which  generally  is  on  a  different  depth  plane.  This  may  also  result  in  possible  eyestrain,  fatigue 
and,  in  many  cases,  inability  to  fuse  the  stereo  image.  Consequently,  a  pair  of  stereoscopic 
cameras  with  a  fixed  separation  and  convergence  angle  can  in  practice  ^-verely  limit  the 
operator’s  ability  to  perform  effectively  over  a  broad  range  of  viewing  conditions  and 
scenarios. 

3.3  Manipulator  Orientation  and  Alignment  Problems 

19.  With  most  military  robotic  platforms,  the  camera  assembly  is  typically  placed 
somewhere  on  the  arm  of  the  robot.  This  placement  of  the  camera  package  prevents  feedback 
of  information  on  the  specific  orientation  of  the  manipulator  with  respect  to  the  platform, 
however.  This  limitation  can  be  a  serious  handicap  for  the  operator,  who  needs  to  be  able  to 
visualise  how  the  various  system  and  environmental  components  are  aligned  with  respect  to 
each  other.  In  some  instances  it  is  possible  for  the  cameras  to  be  situated  on  either  a  different 
manipulator  link,  or  on  a  platform  which  is  not  attached  to  the  arm  at  all.  Nevertheless,  even  in 
these  cases  problems  can  arise,  as  the  operator  must  be  able  to  transform  what  he  sees  from  his 
own  camera  frame  of  reference  to  the  task  frame  of  reference.  For  some  tasks,  such  as  pick  and 
place,  insertion  tasks,  etc.,  critical  errors  may  result  from  this  need  to  work  with  these  different 
frames  of  reference. 

3.4  Operator  Workload 

20.  Accurate  teleoperation  involves  significant  attentional  demands.  The  perceptual 
limitations  of  the  remote  viewing  system  can  intensify  the  severity  of  these  demands.  In 
addition,  the  memory  load  on  the  operator  can  be  quite  high,  in  cases  for  which  retention  of 
control  actions  performed  during  the  mission  can  be  essential  to  identify  the  state,  location,  and 
direction  of  the  remote  system  at  any  stage  in  the  operation.  This  may  be  critical  to  the  success 
of  the  mission,  particularly  when  the  operator  has  little  or  no  direct  visual  contact  with  the 
remote  system.  The  stress  imposed  by  this  requirement  is  further  exacerbated  when  operating 
in  physically  complicated  environments,  under  high  time  pressure,  or  even  when  the  periods  of 
required  vigilance  are  unduly  long,  e.g.  when  controlling  very  slow  systems.  Such  enforced 
attention  requirements  not  only  limit  the  number  of  concurrent  activities  which  the  operator  is 
able  to  handle,  but  can  also  lead  to  increased  likelihood  of  error,  and  may  ultimately  endanger 
the  mission. 


4.  -C-QNFIG.URATIQN  OF  SV+SG  DISPLAY  SYSTEM 

21.  A  generic  block  diagram  of  the  system  under  development  is  given  in  Fig.  1.  It 
consists  of  four  basic  elements,  as  follows: 

•  Stereoscopic  Video  Camera  System, 

•  Stereoscopic  Viewing  System, 

•  Pointer  Positioning  Device, 

•  Control  Computer. 

22.  The  first  and  second  elements  comprise  the  basic  functional  components  of  the 
stereoscopic  video  system,  based  on  alternating-field,  or  time-multiplexed,  stereoscopy.  The 
fundamental  principles  of  this  system  have  been  described  elsewhere  (Milgrara  et  al,  1989, 


STEREO  VIDEO 
CAMERA  SYSTEM 


Fig.  1  Block  Diagram  of  Stereoscopic  Video 
+  Stereoscopic  Graphic  Display  System 
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1990;  Milgram  &  van  der  Horst,  1986).  Briefly,  this  class  of  stereo-pair  displays  involves 
rapidly  alternating  left-  and  right-eye  viewpoint  images  on  a  video  or  computer  monitor.  When 
the  stereoscopic  images  are  generated  as  live  video  pictures,  the  signals  from  each  of  the  left  and 
right  cameras  are  electronically  combined,  such  that  one  is  transmitted  as  the  odd  field  and  the 
other  as  the  even  field  of  a  single  interlaced  video  signal.  When  this  signal  is  viewed  directly  on 
the  monitor,  it  is  perceived  as  a  flickering  double  image.  When  viewed  through  special  puipose 
synchronised  liquid  crystal  spectacles,  however,  a  single  stereoscopic  image  is  perceived.  This 
is  because  the  spectacles  act  as  a  shuttering  device,  which  alternately  blocks  and  unblocks  the 
view  of  the  display  from  each  eye,  in  synchrony  with  the  alternating  display  images.  The  result 
is  that  the  left  eye  is  blocked  whenever  the  right  eye  image  is  displayed  and  unblocked 
whenever  the  left  eye  image  is  presented,  and  vice  versa. 

23.  Stereoscopic  images  can  be  created  by  a  computer  using  a  similar  technique.  The 
stereographic  image  generation  system,  shown  in  Fig.  1  as  a  sub-function  of  the  Control 
Computer,  is  responsible  for  generating  pairs  of  alternating  images  which  are  analogous  to 
corresponding  left  and  right  camera  images. 

24.  The  unique  aspect  of  our  system,  as  described  thus  far,  is  its  ability  to  combine 
stereoscopic  video  (SV)  and  stereoscopic  graphic  (SG)  images  and  view  them  on  a  single 
monitor.  This  is  accomplished  with  the  Mixing  Device,  shown  in  Fig.  1,  which  combines  the 
synchronised  video  signals  from  both  the  computer  and  the  cameras.  The  net  result  is  a  three 
dimensional  image  within  which  virtual  SG  objects  generated  by  the  computer  appear  within  the 
live  SV  video  world. 

25.  An  illustration  of  the  capabilities  outlined  thus  far  is  given  in  Fig.  2,  in  which  a  park 
scene,  on  the  left  hand  side,  is  depicted  as  being  viewed  via  the  (dual  camera)  SV  system.  The 
reader  is  requested  to  imagine  that  this  scene,  observed  with  the  help  of  the  shuttering 
spectacles,  is  perceived  stereoscopically  on  the  monitor,  shown  on  the  right  hand  side.  The 
tree,  the  bench,  the  dustbin,  the  rock  and  the  box  all  represent  "live"  objects,  which  are 
provided  by  the  video  cameras.  The  virtual  objects  in  the  picture,  that  is,  the  "V"  shaped 
triangles  and  the  line  joining  them,  as  well  as  the  second  box  drawn  with  dashed  lines,  are 
generated  by  the  SG  computer  and  appear  to  the  observer  to  be  actually  present  among  the  real 
objects  in  the  video  world. 

26.  Because  they  are  computer  generated,  it  is  clearly  possible  to  cause  the  virtual  SG 
objects  to  move  around  freely  within  the  SV  world,  by  making  use  of  the  Pointer  Positioning 
Device  (a  three  degree  of  freedom  trackball  in  our  case),  indicated  schematically  in  Fig.  1.  This 
is  accomplished  by  scaling  the  size  and  position  in  depth  of  a  superimposed  SG  object  so  that  it 
appears  realistically  within  the  SV  scene.  This  means  that,  if  a  particular  object  is  drawn 
graphically  and  then  moved  to  the  same  location  in  (x,y,z)  space  as  a  corresponding  real  world 
object  with  identical  dimensions,  the  two  should  coincide  exactly  on  the  screen.  To  accomplish 
this  all  virtual  images  are  calibrated  so  that  any  (x,y,z)  input  from  the  pointer  positioning  device 
is  interpreted  directly  in  terms  of  real  world  units.  The  physical  parameters  which  enter  into 
those  calculations  include  tire  following: 

•  camera  separation, 

•  camera  convergence  angle, 

•  alignment  of  camera  optics, 

•  positions  of  the  centroids  of  die  lenses, 

•  mapping  between  the  video  sensors  and  die  computer  display. 


Fig.  2  Example  of  "Virtual  Pointer"  +  Overlaid  Wireframe  Image 


27.  If  all  of  the  parameters  listed  above  have  been  computed  or  measured  correct1  y, 
system  validity  can  be  verified  by  superimposing  and  visually  comparing  SG  object  images 
with  corresponding  real  objects.  For  our  purposes,  it  is  sufficient  that  these  be  aligned  only  to 
an  extent  that  is  within  the  resolution  capabilities  of  the  visual  display  system.  In  order  to 
enhance  the  overall  reliability  of  the  system,  however,  we  are  currently  extending  it  by  adding 
frame  grabbing  hardware,  as  shown  in  Fig.  1,  which  will  serve  as  a  basic  component  of  our 
ultimate  "virtual  control"  capability.  This  and  other  functional  capabilities  are  discussed  in  the 
following  section. 


5.  EUNCTIONAL  CAPABILITIES  -  CURRENT  AND  FUTURE 

28.  In  this  section  we  refer  to  the  operational  issues  outlined  in  section  3  and  discuss 
how  the  current  functional  capabilities  of  our  SV+SG  display  system  address  those  problems. 
In  addition,  we  present  our  planned  enhancements  of  the  system,  and  discuss  their  relevance  to 
practical  telerobotic  operations. 

5.1  Stereographic  Pointer  /  Tape  Measure  Capability 

29.  In  Section  3.1  operational  aspects  of  the  problem  of  being  able  to  make  absolute 
distance  or  location  measurements  are  outlined.  That  problem  has  been  the  principal  motivation 
for  our  development  of  the  virtual  pointer  capability.  Because  humans  are  very  sensitive  at 
making  relative  discriminations,  but  are  not  very  good  at  making  absolute  judgements  (van  Cott 
&  Warrick,  1972),  it  is  logical  to  try  and  convert  all  such  estimation  tasks  to  relative 
discrimination  tasks,  wherever  possible.  This  is  the  same  general  principle  that  is  employed  in 
supplying  anchoring  stimuli  in  practical  display  design,  to  assist  people  in  absolute  judgement 
tasks  (Wickens,  1984). 

30.  The  stereographic  pointer  is  generated  by  the  graphics  computer  and  can  be  moved 
throughout  the  three  dimensional  space  by  the  multi-degree  of  freedom  Pointer  Positioning 
Device.  The  pointer  must  be  readily  visible  on  the  screen  and  must  have  an  obvious 
characteristic  single-point  vertex  for  alignment  purposes.  Pointer  formats  with  which  we  have 
experimented  thus  far  have  included  cross-hairs,  an  open  or  filled  letter  V  (as  shown  in  Figure 
2)  and  upward  or  downward  facing  arrows.  It  has  become  clear  that  the  "best"  pointer  format 
should  be  determined  by  the  particular  operational  circumstances,  including  such  factors  as 
predominant  orientation  and  density  of  objects  at  the  remote  site,  ambient  illumination,  etc.  The 
great  advantage  of  employing  a  computer  generated  tool  such  as  this,  of  course,  is  that  pointer 
shape  and  orientation,  as  well  as  brightness  and  colour,  can  be  selected  by  the  operator  during 
usage,  to  suit  the  situation. 

3 1 .  The  three  dimensional  pointer  location  on  the  screen  is  calibrated  in  absolute  real- 
world  units,  as  a  function  of  the  basic  physical  characteristics  of  the  stereoscopic  video  system, 
which  are  listed  in  Section  4.  Using  the  Pointer  Positioning  Device,  the  pointer  can  be  aligned 
with  any  feature  of  the  real-world  scene.  The  corresponding  real-world  position  coordinates 
(x,y,z)  can  then  be  computed  and,  using  the  SG  option,  can  even  be  displayed 
alphanumerically  at  the  same  depth  plane  of  the  pointer  in  3-D  space. 
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32.  If  two  such  points  are  identified  at  the  remote  site,  the  distance  between  them  is 
readily  calculated.  This  is  the  basis  of  the  “ tape-measure ”  caoability,  which  is  graphically 

depicted  in  Figure  2.  To  implement  it,  the  viewer/operator  triggers  the  pointer  at  one  location  , 

and  “drags”  a  virtual  line  along  a  three  dimensional  trajectory  to  a  second  point  in  the  scene. 

The  computer  then  calculates  and  displays  or  announces  the  absolute  distance  between  the  two 
registered  points,  as  illustrated. 

5.2  Hybrid  Pointer  Alignment  System 

33.  As  emphasised  above,  the  capability  of  making  absolute  3-D  measurements  of  target 
locations  is  potentially  very  useful  in  practical  telerobotics.  It  is  important  to  recognise, 
however,  that,  using  the  SG  pointer  alone,  the  final  measurement  is  subject  to  two  possible 
sources  of  error.  One  derives  from  the  procedure  for  calibrating  the  SG  software  relative  to  the 
camera  system,  while  the  other  is  the  result  of  the  operator’s  limitations  in  matching  the 
perceived  location  of  the  virtual  pointer  with  the  real-world  location  of  target  objects. 

34.  In  response  to  the  second  problem,  the  SG  pointer  concept  is  being  incorporated  into 
an  enhanced  hybrid  pointer  alignment  system,  involving  a  higher  level  of  automation.  The 

advanced  SG  pointer  system,  which  is  being  implemented  on  a  more  sophisticated  graphics  ** 

workstation,  makes  use  of  frame  grabbing  hardware  that  allows  digital  recording  of  the  real- 

world  scene.  This  will  facilitate  computational  analyses,  including  machine  vision  detection  of 

edges  or  comers,  to  be  used  for  objective  determination  of  location,  dimensions  and  orientation 

of  objects  in  the  environment. 

35.  This  system  will  enable  the  operator  to  use  the  SG  pointer  to  perform  initial  j 

designation  of  points  or  objects  of  interest  and  then  allow  the  computer  to  cany  out  automated 

processing  to  generate  an  objective  image  of  the  remote  scene  in  the  vicinity  of  the  SG  pointer. 

The  display  system  will  then  communicate  to  the  operator,  also  by  means  of  overlaid  graphics. 
its  version  of  which  points  or  objects  it  believes  the  operator  has  selected.  If  this  does  not 
coincide  with  the  operator's  intention,  the  procedure  is  repeated  until  the  two  system 
components,  the  human  and  the  computer,  come  to  an  agreement 

36.  The  philosophy  behind  this  hybrid  system  is  to  utilise  optimally  the  strengths  of  both 
the  human  and  the  computer.  With  the  original  SG  pointer  system  outlined  in  Section  5.1,  even 
though  the  computer  knows  where  in  the  real  world,  relative  to  the  video  cameras,  it  has  drawn 
the  pointer,  the  computer  possesses  no  knowledge  at  all  about  what  is  in  the  real  world  being 
viewed  by  the  video  cameras.  The  human  must  therefore  perform  the  high  level  function  of 

recognising  and  designating  operationally  strategic  targets,  as  well  as  the  low  level  function  of  r 

precisely  aligning  tire  SG  pointer  with  those  objects.  With  the  enhanced  system,  the  human  will 
be  spared  a  portion  of  the  low  level  aligning  task.  Even  though  the  computer  will  still  have  no 
knowledge  of  what  is  in  the  real  world  (which  is  an  important  contradistinction  with  most 
Artificial  Intelligence  applications),  it  will  have  access  to  detailed  quantitative  information  about 
where  things  are  in  that  world.  This  will  permit  the  computer  to  perform  the  low  level 
computations  necessary  for  specifying  points  in  space,  while  the  human  operator  is  able  to 
concentrate  more  on  strategic  planning  and  decision  making. 

37.  It  is  important  to  point  out  that  this  process  will  reduce,  but  not  completely  eliminate, 
die  operator's  labour-intensive  task  of  manually  aligning  the  virtual  SG  pointer  with  real-world 
points  in  the  3-D  video  space.  What  it  will  provide,  however,  will  be  a  faster  and  more  reliable 
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system,  which  will  result  in  more  efficient  teleoperation,  with  a  significantly  reduced  operator 
workload ',  which  in  turn  will  permit  the  operator  to  turn  her  attention  to  other  concurrent  control 
activities. 

5.3  "Virtual  Control" 

38.  The  logical  extension  of  the  hybrid  system  outlined  above  is  one  which  permits 
"virtual  control”  of  the  telerobot.  What  this  involves  is  a  system  which  enables  the  human 
operator  to  rise  above  the  low  level  function  of  continuous  manual  control  of  the  teleoperator, 
and  allows  her  instead  to  designate  intermediate  robot  states,  or  "way  points",  which  can 
subsequently  be  achieved  by  the  control  computer.  This  concept  is  consistent  with  Sheridan's 
well  known  model  of  supervisory  control  of  teleroiic  systems  (Sheridan,  1984).  A  similar 
concept  has  been  proposed  by  Conway  et  al  (1990),  using  a  somewhat  different  technological 
approach. 

39.  As  a  simple  example  of  virtual  control,  the  pointer  could  be  used  to  design  a 
collision-free  pathway  for  a  teleoperated  vehicle,  which  would  then  be  programmed  to  follow 
that  pathway.  An  example  of  such  an  application  is  shown  in  Fig.  3.  In  the  same  context  as 
Fig.  2,  the  operator  has  used  the  pointer  here  to  designate  to  the  control  computer  a  set  of  way- 
points  along  a  desired  trajectory.  In  the  figure  the  computer  is  also  displaying  stereoscopically 
to  the  operator  how  this  3-D  trajectory  will  appear  within  the  real-world  site. 

40.  With  knowledge  of  certain  dimensional  parameters  of  the  robot,  this  pathway 
planning  function,  together  with  the  local  world  mapping  function,  could  become  more 
sophisticated;  that  is,  the  computer  could  cam'  out  checks  on  the  operator-designated  pathway 
to  ensure  that  vehicle-obstacle  clearances  allow  for  a  collision-free  route.  Path 
smoothing/optimisation  is  another  possibility.  The  same  principles  and  techniques  could  be 
applied  to  manipulator  movement/path  planning  in  a  congested  environment.  Another  important 
application  domain  involves  systems  with  very  long  dynamic  time  constants  and/or  with 
significant  time  delays.  In  all  cases,  the  operator's  control  workload  should  over  the  long  term 
be  reduced  considerably,  allowing  more  attention  to  be  allocated  to  high  level  planning  and 
decision  making  tasks. 

5.4  Generalised  Stereographic  Superposition 

41.  The  3-D  graphics  techniques  used  to  generate  the  SG  pointer  described  above  have 
been  extended  to  allow  generation  of  more  complex  wireframe  images,  which  can  be  drawn 
both  stereoscopically  and  in  perspective.  This  general  capability  is  illustrated  by  the  wireframe 
box  drawn  with  dashed  lines  at  the  bottom  of  the  right  hand  side  of  Fig.  2. 

42.  Data  for  such  three-dimensional  wireframe  images  can  be  derived  from  digital 
analysis  of  real-world  objects  or  from  available  computer  models  of  those  objects.  The  images 
of  the  targets  can  then  be  superimposed  onto  the  real-world  video  scene  to  simulate  or  enhance 
target  visibility.  This  capability  could  be  particularly  useful,  for  example,  for  teleoperation  in 
the  vicinity  of  partially  visible  or  partially  obscured  objects  with  known  characteristics,  which 
permits  generation  of  an  accurate  virtual  wireframe  construct.  It  could  also  be  useful  for 
teleoperation  with  totally  invisible  objects  with  known  characteristics  in  a  structured  or  partially 
structured  environment. 
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Fig.  3  "Virtual  Pathway"  Example 
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43.  Such  wireframe  images  can  be  also  used  to  generate  virtual  objects  and  thereby  create 
totally  virtual  scenario.,.  This  “simulation”  capability  is  anticipated  to  be  most  useful  for 
training  purposes,  including  safe  and  effective  teleoperation  in  a  complex,  partially  structured 
environment.  Wireframe  constructs  could  be  used  in  such  cases  to  provide  graphic  displays, 
for  example  of  a  recommended  trajectory  for  a  teleoperated  vehicle  in  an  environment  with 
obstacles.  A  one-dimensional  example  of  this  is  shown  in  Fig.  3;  however,  there  is  no  reason 
why  more  complex  formats,  such  as  heads-up  "path-in-the-sky"  types  of  displays  (Stokes  et  al, 
1990)  could  not  be  used  here. 

44.  One  potential  application  of  this  technique  is  for  simulating  and  displaying  predicted 
manipulator  movements  or  predicted  projectile  trajectories.  When  combined  with  the  “spread” 
characteristics  of  the  projectile,  for  example,  it  could  also  be  used  to  display  a  “cone  of  danger” 
associated  with  the  weapon.  Another  anticipated  application  is  as  a  means  of  graphically 
superimposing  coordinate  reference  frames,  which  can  be  used  in  the  context  of  a  heads-up 
display  for  complex  alignment  operations  in  multi-degree-of-freedom  telemanipulation  tasks. 

5.5  Dynamic  Optimisation  of  SV  Camera  Configuration 

45.  It  was  shown  in  section  3.2  above  that  a  pair  of  fixed  base  stereoscopic  cameras  can 
potentially  limit  the  effectiveness  of  teleoperations,  especially  if  surveillance  or  manipulations 
are  to  be  carried  out  within  a  relatively  large  dynamic  range,  or  if  enhanced  depth  resolution  is 
necessary  for  precision  or  detection  purposes.  However,  adjustment  of  camera  separation  and 
angle  of  convergence  to  match  the  observer’s  focus  of  attention  and  operational  needs  is  a  time- 
consuming  and  inexact  procedure.  Although  such  a  function  is  clearly  a  prime  candidate  for 
automation,  the  difficulty  of  making  such  adjustments  automatically,  and  especially 
dynamically,  i.e.  on-line,  is  compounded  by  the  necessity  of  communicating  to  the  control 
computer  what  the  observer's  operational  needs  are  and  where  the  observer's  focus  of  attention 
is.  In  response  to  this  issue,  an  automated  dynamic  camera  base  optimisation  system  is  being 
implemented. 

46.  Using  the  arrangement  shown  in  Fig.  4,  the  cameras  are  mounted  on  supports  whose 
orientation  is  controlled  by  two  roman  screws,  each  with  both  left  and  right  hand  threads, 
allowing  symmetrical  translations  of  the  camera  supports.  Simultaneous  rotation  of  the  screws 
is  accomplished  through  the  use  of  motors  controlled  from  the  computer  supporting  the  vision 
system.  Based  on  a  particular  region  of  operational  interest,  either  prescribed  independently  or 
indicated  on-line  via  the  SG  pointer,  the  computer  is  able  to  adjust  the  camera  alignment  to 
optimise  the  depth  range  and  resolution  of  the  stereoscopic  video  system.  A  similar  procedure 
can  be  used  to  control  the  focus  and  zoom  of  the  camera  lenses. 


6.  CONCLUSION 

47.  The  SV+SG  display  system  described  here  was  implemented  originally  using  an 
Amiga  2500  as  the  graphics  computer.  That  platform  is  currently  being  transferred  to  a  Silicon 
Graphics  4D/310  graphics  workstation,  which  will  permit  near  real-time  animation  of  complex 
solid  images,  in  addition  to  the  wireframe  images  discussed  above.  In  this  section  we  outline  a 
number  of  operational  issues  experienced  thus  far  with  the  SV+SG  display  system  described 
above. 


AC/243-TP/3 


C2.14 
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Hand  Threads 


Fig.4  Schematic  Diagram  of  Camera  Drive  System 
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48.  The  setting  up  and  use  of  stereoscopic  video  for  teleoperation  is  associated  with  a 
number  of  specific  problems  associated  with  the  system  itself.  Many  camera  models  are 
unsuitable  for  use  in  a  stereoscopic  pair  due  to  their  large  physical  size.  This  makes  it  difficult 
to  obtain  orthostereoscopic  conditions,  which  generally  corresponds  to  that  which  is  perceived 
by  an  average  human,  with  an  interpupillary  distance  of  about  62  mm.  In  addition, 
commercially  available  video  cameras  are  typically  not  designed  or  manufactured  to  meet  other 
stringent  requirements  of  satisfactory  performance  in  a  stereoscopic  pair  system.  For  example, 
most  cameras  exhibit  significant  variations  in  the  offset  of  the  optical  axis  of  the  sensing  element 
with  respect  to  direction  and  point-of-intersection  at  the  sensor  and  the  optical  axis  of  the  lens 
system  and  the  body  of  the  camera.  If  the  variations  are  slight,  this  problem  is  not  a  major  one 
for  stereoscopic  video  alone.  However,  when  mixing  stereoscopic  video  (SV)  with 
stereoscopic  computer  graphics  (SG),  this  "nonmisalignment  problem"  can  have  major 
detrimental  effects. 

49.  A  preliminary  experiment  has  been  conducted  to  evaluate  the  accuracy  with  which  the 
SG  pointer  can  be  aligned  with  targets  in  the  real  world  (Drascic  &  Milgram,  1991).  This  work 
has  found  that  the  pointer  can  be  positioned  with  a  precision  close  to  human  stereoacuity 
perceptual  limits  (approximately  20  seconds  of  arc  disparity),  and  with  a  reliability  comparable 
to  that  of  aligning  a  real  pointer  remotely.  (The  stereographic  pointer  has  a  standkd  deviation 
of  approximately  90  seconus  of  arc  disparity,  while  the  real  pointer  had  a  standard  deviation  of 
approximately  60  seconds  of  arc  disparity.) 

50.  The  stereoscopic  video  camera  mount  described  above  has  been  constructed  and 
interfaced  to  the  micro-computer.  Software  to  permit  the  manual  control  of  the  separation  and 
convergence  angles  of  the  cameras  has  been  developed.  The  heuristics  necessary  for 
automatically  adjusting  the  camera  parameters  based  on  the  operator’s  control  of  the  SG  pointer 
have  been  specified  and  are  currently  being  implemented.  An  experimental  investigation  of  a 
number  of  operational  tradeoffs  among  various  camera  configuration  parameters  is  about  to 
begin. 

51.  Software  has  been  developed  to  generate  arbitrary  stereoscopic  wireframe  graphics 
from  a  database  of  objects.  These  wireframes  can  be  animated  to  indicate  moving  items. 
Planned  for  the  near  future  are  graphics  tools  to  aid  in  the  creation  of  the  object  database.  The 
objective  is  for  the  animating  computer  to  be  able  to  generate  an  appropriate  wireframe  image  of 
a  particular  object  in  any  location  and  with  any  orientation  within  the  remote  view.  The 
minimum  specification  for  this  function  is  that  it  be  possible  for  the  SG  object  to  be  perfectly 
superimposed  upon  a  corresponding  real-world  object  of  tire  same  dimensions. 
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M.  Abstract:  — 

The  Human  Engineering  Laboratory  (HEL),  in  partnership  with  the 
Combat  Systems  Test  Activity,  is  developing  facilities  at  Aberdeen 
Proving  Ground  to  support  research  and  test  robotic  systems.  These 
facilities  include  an  indoor  and  outdoor  test  couse  instrumented  for 
automatic  data  collection.  A  remotely-controlled  golf  cart  is 
among  the  tools  used  by  the  HEL  to  study  human  factors  design  and 
interface  issues  associated  with  teleoperation.  This  platform,  along 
with  a  teieoperated  High  Mobility  Multi-Purpose  Wheeled  Vehicle,  is 
currently  being  used  in  a  program  of  research  in  low  data  rate 
driving.  This  program  seeks  to  optimize  soldier/operator 
performance  under  tactical  conditions  requiring  reduced  bandwidth 
transmissions.  During  this  research  the  influence  of  various  image 
compression  and  enhancement  techniques  on  remote  driver  performance 
will  be  measured. 
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REMOTE  OPERATION  USING  A  LOW  DATA  RATE  COMMUNICATIONS 

LINK 


1.1  INTRODUCTION 

1.  Although  the  technology  for  constructing  robotic 
systems  exists,  the  knowledge  base  needed  to  support 
cost-effective  design  decisions  is  still  lacking.  Current 
research  in  robotics  at  the  Human  Engineering  Laboratory 
(HEL)  is  geared  toward  identifying  visual  display  and  control 
device  design  characteristics  required  for  teleoperation, 
particularly  as  they  apply  to  the  quantity  and  quality  of 
sensory  input  actually  needed  by  the  remote  operator  to 
perform  a  given  task  effectively.  This  paper  describes  some 
of  the  tools  and  facilities  developed  to  support  these 
current  study  efforts  as  well  as  a  program  of  research  in  low 
data  rate  remote  driving  called  ROAD  RUNNER. 

1.2  RESEARCH  FACILITIES 

2.  In  November  1988,  with  the  signing  of  a  Memorandum 
of  Understanding,  the  HEL  and  the  U.S.  Army  Combat  Systems 
Test  Activity  (USACSTA)  embarked  on  a  cooperative  endeavor  to 
establish  facilities  at  Aberdeen  Proving  Ground  that  would 
support  the  Army's  effort  to  effectively  apply  robotic 
technology  to  combat,  combat  support,  and  combat  service 
support  materiel.  Since  that  time,  several  indoor  and  outdoor 
test  courses  have  been  developed  specifically  to  support 
research  and  test  of  robotic  systems.  One  such  course, 
located  within  a  former  aircraft  hangar,  provides  a 
controlled  environment  for  assessment  of  remotely  operated 
vehicles.  Another  outside  course  enables  the  researcher  to 
collect  similar  data  under  more  variable,  real-world 
conditions.  Beth  courses  are  instrumented  to  facilitate 
automatic  data  collection. 
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1.2.1  Indoor  Test  Course 

3 .  The  indoor  test  course  is  housed  in  a 
3200-square-meter  aircraft  hangar.  Its  black  macadam  roadway 
is  2.7  meters  wide  and  approximately  400  meters  long  (see 
Figure  1) .  The  area  surrounding  the  road  is  painted  a  lighter 
shade  to  define  path  boundaries.  The  course  consists  of  six 
segments  that  include  straightaways,  serpentine,  right-  and 
left-hand  turns,  a  figure  8,  and  an  obstacle  avoidance 
segment.  Driving  performance  on  each  of  these  segments  is 
scored  automatically,  and  summary  statistics  are  available 
immediately  after  each  run.  These  performance  data  include 
vehicle  speed  and  accuracy. 


90  m 
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Figure  1.  CSTA-HEL  Robotic  Test  Facility 
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4.  The  measure  of  accuracy  for  all  course  segments, 
except  for  the  obstacle  avoidance  segment,  is  the  amount  of 
absolute  deviation  from  the  centerline  of  the  road.  This 
centerline,  along  with  four  other  stripes,  are  painted  on  the 
roadway's  surface.  Each  stripe  is  approximately  1.3 
centimeters  wide.  The  stripes  are  spaced  68.5  centimeters 
apart  and  run  parallel  along  the  length  of  the  course.  A 
fluorescent  light,  video  camera,  and  transmitter  are  mounted 
within  a  hood  attached  to  the  front  of  the  vehicle.  The 
fluorescent  light  illuminates  the  stripes  on  the  road 
directly  beneath  the  hood  for  the  video  camera  (see  Figure 
2) .  The  video  image  of  these  stripes  is  transmitted  to  the 
data  acquisition  center  for  processing  by  two  contrast 
trackers.  These  trackers  lock  onto  the  right  edge  of  the 
right-most  stripe  in  the  f ield-of-view  (FOV)  of  the  camera 
and  compute  the  position  of  that  stripe  relative  to  the 
camera's  horizontal  FOV.  Data  pertaining  to  deviations  from 
road  centerline  can  be  collected  within  2  to  5  centimeters  of 
accuracy  at  a  rate  of  60  times  a  second.  This  measure  also 
enables  computation  of  distance  traveled  with  one,  two, 
three,  or  four  wheels  off  the  road. 


Figure  2.  On-board  instrumentation  for  measurement  of 
deviations  from  road  centerline. 


5.  The  obstacle  avoidance  segment,  located  at  the  end 
of  the  course,  is  the  last  maneuver  to  be  performed.  The 
number  of  pylons  hit  is  used  to  determine  the  level  of 
accuracy  for  this  segment.  These  data  are  provided  by 
normally  open  contact  switches  which  are  incorporated  into 
each  pylon  and  linked  to  the  computer. 

6.  A  microswitch,  located  at  the  start  of  the  course, 
senses  the  commencement  of  a  run  and  data  collection  is 
initiated  automatically.  Data  collection  is  terminated  in  a 
similar  manner.  Microswitches  are  also  located  at  the 
beginning  and  end  of  each  course  segment.  If  the  vehicle 
should  temporarily  stray  off  the  course  to  a  point  where 
there  are  no  stripes  within  the  FOV  of  the  camera, 
microswitches  located  every  4.9  meters  within  each  segment 
identify  the  vehicle  location  upon  its  return  and  reinitiate 
data  collection.  Vehicle  speed  can  be  computed  within  each  of 
these  intervals  based  on  time  and  distance  traveled.  A  fifth 
wheel  is  used  to  provide  an  accurate  measure  of  distance 
traveled. 

1.2.2  Outdoor  Test  Course 

7.  The  Rolling  Road  course  is  one  of  several  outdoor 
courses  that  comprise  the  60.7-hectare  Munson  Test  Range  at 
Aberdeen  Proving  Ground  (see  Figure  3).  The  course  is  3.7 
meters  wide  and  approximately  2  kilometers  long.  It  is 
composed  of  a  variety  of  terrain  features  including  gravel 
and  macadam  road  surfaces,  open  and  tree-lined  areas,  flat 
and  rolling  terrain,  and  assorted  straight  and  twisting 
paths.  The  course  consists  of  three  segments.  One  segment 
includes  left  and  right  turns  of  varying  severity.  The  two 
remaining  segments  are  both  straightaways.  One  straightaway 
is  composed  of  a  flat,  macadam  surface;  the  other  is  more 
rugged  rolling  terrain  consisting  of  hard  packed  gravel  with 
numerous  potholes.  These  straightaways  are  of  equal  length. 
Three  types  of  obstacles,  six  of  each  type,  are  located  at 
surveyed  points  throughout  the  course.  The  three  types  of 
obstacles  used  are  pylons,  railroad  ties,  and  man-made  rocks. 


,  i  l 

The  course  is  reconfigured  by  changing  the  location  of  the  i  :  \ 

obstacle  types.  For  all  configurations,  the  obstacles,  are  .  j 

equally  distributed  on  both  left-  and  right-hand  sides  of  the  p.  J 

roadway  and  the  number  of  each  type  of  obstacle  remains  the  i 

same  within  each  course  segment .  1/ 
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8.  Driver  performance  is  scored  based  on  vehicle  speed, 
deviations  from  the  "perfect  path",  and  obstacles  hit.  The 
"perfect  path"  is  defined  as  the  surveyed  centerline  of  the 
road  and  the  minimum  deviation  the  vehicle  must  take  from 
this  centerline  to  avoid  hitting  an  obstacle  (see  Figure  4) . 
The  Radio  Frequency  Navigational  Grid  (RFNG) ,  developed  by 
Kaman  Sciences  Corporation,  is  used  to  locate  the  position  of 
the  vehicle.  These  data  are  used  to  compute  vehicle  speed  and 
deviations  from  the  "perfect  path"  as  well  as  the  number  of 
obstacles  hit.  Position  location  is  obtained  by  phase 
differencing  using  a  hyperbolic  grid.  The  RFNG  can  compute 
the  X  and  Y  coordinates  of  the  vehicle  at  a  rate  of  one  per 
second  and,  according  to  the  manufacturer,  can  locate  the 
position  of  the  vehicle  within  10  to  15  centimeters.* 


Figure  4.  The  perfect  path 


*  Validation  tests  are  under  way  to  ascertain  position 
location  accuracy. 
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9.  Vehicle  position  location  data,  provided  by  the 
RFNG,  are  also  used  to  estimate  the  distance  from  the 
obstacle  upon  its  identification  by  the  test  participant. 

This  technique  employs  a  data  collector  who  depresses  a 
button  upon  hearing  a  correct  verbal  identification  from  the 
test  participant.  This  button  is  linked  with  the  computer 
which  flags  the  location  of  the  vehicle  at  that  moment  during 
the  run  and  calculates  its  distance  from  the  obstacle. 

10.  Two  additional  driving  segments  are  located  just 
off  the  main  course.  They  include  an  obstacle  avoidance 
segment  similar  to  that  of  the  indoor  test  course,  and  a 
segment  called  "parking."  As  on  the  indoor  test  course,  the 
obstacle  avoidance  segment  is  comprised  of  a  row  of  equally 
spaced  pylons.  The  remote  driver's  task  is  to  maneuver  the 
vehicle  between  and  around  these  pylons  as  quickly  as 
possible  without  hitting  them  (see  Figure  5)  .  Driving 
performance  is  scored  based  on  time  to  complete  and  the 
number  of  pylons  hit.  The  "parking"  segment  is  also  comprised 
of  pylons  arranged  to  form  a  rectangle  with  an  opening  at  one 
end  (see  Figure  6) .  In  this  segment,  the  remote  driver  must 
maneuver  the  vehicle  into  the  rectangle  without  hitting  any 
of  the  pylons  and  stop  as  close  to  those  which  form  the  rear 
of  the  enclosure  as  possible.  Performance  measures  include 
time  to  complete  and  the  number  cf  pylons  hit.  Normally  open 
contact  switches  are  used  to  identify  pylons  hit  in  both  of 
these  driving  tasks. 
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Figure  6.  Parking  segment. 


.11.  The  Road  Runner  vehicle  is  one  among  a  number  of 
remotely  controlled  platforms  currently  being  used  by  the  HEL 
to  study  human  factors  design  and  interface  issues  associated 
with  teleoperation.  These  platforms  range  in  size  from  the 
smaller  explosive  ordnance  disposal  vehicles  to  the  High 
Mobility  Multi-Purpose  Wheeled  Vehicle  (HMMWV) .  The  Road 
Runner  is  a  simple,  low  cost,  highly  reliable  teleoperated 
golf  cart  converted  for  remote  operation  for  the -  HEL  by  '• 
Tooele  Army  Depot  in  Salt  Lake  City,  Utah  (see  Figure  7) .  ■  -■ 

This  vehicle  was  developed  specifically  to  serve  as  a 
research  platform  for  examining  control  and  display  design 
and  information  requirements  for  teieoperat ion .  The  vehicle 
is  powered  by  six  6-volt  rechargeable  batteries  and  is 
capable  of  speeds  of  up  to  19  kilometers  per  hour. 
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12.  The  vehicle  controls  consist  of  a  steering  wheel, 
brake,  and  accelerator  pedals.  Information  on  vehicle  speed, 
wheel  direction,  and  system  voltage  are  transmitted  front  the 
Road  Runner  vehicle  and  displayed  on  dial-type  gauges  at  the 
operator's  remote  control  panel.  The  vehicle  is  capable  of 
being  operated  from  an  on-board  driving  position  as  well  as 
remotely  using  the  same  control  station.  In  the  remote 
driving  mode,  the  vehicle's  control  station  is  seated  within 
a  frame  containing  both  a  12-volt  battery  pack  and  electrical 
connector.  The  control  station  is  attached  to  the  frame  by 
four  bolts  and  a  power  hook-up  cable.  The  station  is  easily 
removed  as  a  unit  from  the  frame  and  reinstalled  on  board  the 
vehicle.  This  unique  design  feature  enables  the  researcher  to 
measure  and  more  reliably  compare  on-board  versus. remote 
driving  performance. 

13.  The  Road  Runner  incorporates  numerous  safety 
features.  These  include  front  and  rear  bumpers  which,  upon 
contact  with  an  object  (8.9  newtons  of  force  or  more),  shut 
down  the  motor  and  apply  the  brakes.  The  brakes  are  fully 
activated  within  1.5  seconds.  A  large  emergency  stop  button 
on  the  instrument  panel  to  the  right  of  the  steering  wheel 
performs  the  same  function.  The  vehicle  can  be  restarted  by 
depressing  the  initialize  button  to  the  left  of  the  steering 
wheel.  Shutdown  procedures  will  also  be  initiated 
automatically  if  for  any  reason  the  radio  frequency  signal  to 
the  vehicle  is  interrupted.  If  the  throttle  should  stick, 
application  of  the  brakes  will  remove  the  signal  to  the 
motor.  The  brake  and  throttle  must  be  fully  released  before 
the  system  can  be  reactivated. 

14.  Since  its  arrival  at  the  HEL  in  August  1989,  the 
Road  Runner  has  provided  numerous  visitors  hands-on 
experience  in  remote  driving,  both  indoors  and  out.  It  has 
proven  to  be  a  sturdy  and  highly  reliable  platform.  The 
vehicle  is  currently  being  employed  in  a  research  program  in 
low  data  rate  driving  similarly  titled  ROAD  RUNNER. 
Verification  and  extrapolation  of  the  performance  data  base 
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generated  with  this  research  platform  will  be  obtained 
through  selective  field  trials  with  teleoperated  tactical 
vehicles  such  as  the  Army's  HMMWV.  * 


15.  The  ROAD  RUNNER  program  is  one  of  many  HEL  research 
efforts  examining  human  factors  design  and  interface  issues 
associated  with  teleoperation.  The  ROAD  RUNNER  was 
established  to  support  the  Robotic  Testbeds 'program  -  a  major 
cooperative  effort  among  laboratories  within  the  U.S.  Army 
Laboratory  Command  (LABCOM) .  The  HEL  has  the  lead  role  for 
Robotic  Testbeds  within  LABCOM.  The  major  objective  of 
Robotic  Testbeds  is  to  demonstrate  practical  supervisory 
control  of  robotic  combat  vehicles  operating  with  realistic 
mission  requirements,  including  use  of  a  low  data  rate 
tactical  communications  link  for  command  and  control  of  the 
remote  platform. 

16.  Although  fiber  optic  tethers  are  envisioned  to 
serve  as  the  primary  communications  link  between  the 
teleoperator  and  the  remote  platform,  their  survivability  on 
the  battlefield  remains  a  concern.  If  the  fiber  optic  link  is 
severed,  a  secondary  radio  frequency  (RF)  link  would  enable 
the  teleoperator  to  retrieve  the  remote  platform  or  continue 
its  mission.  RF  transmission  of  a  standard  broadcast  quality 
black  and  white  video  image  from  the  remote  vehicle  to  the 
teleoperator,  however,  requires  a  wide  communications 
bandwidth  of  approximately  62  megabits  per  second.  If  RGB 
colour  and  stereovision  are  added,  the  data  rate  requirement 
is  more  than  tripled.  High  data  rate  RF  transmissions  would 
consume  most  of  the  bandwidth  allocated  for  communications  on 
the  battlefield  and  preclude  use  of  existing  tactical  radio 
systems . 

17.  The  ROAD  RUNNER  program  seeks  to  optimize 
soldier-operator  performance  under  tactical  conditions 
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requiring  reduced  bandwidth  transmissions .  The  primary 
objective  of  this  research  is  to  minimize  the  time  to  emplace 
a  remotely  driven  ground  vehicle  while  minimizing  the 
bandwidth  required  for  transmitting  the  video  imagery  to  the 
remote  driver.  Our  bandwidth  goal  is  16  kilobits  per  second. 
This  goal  corresponds  to  the  bandwidth  capabilities  of  the 
Single  Channel  Ground-Air  Radio  System  (SINCGARS) ,  a  secure 
tactical  radio  which  is  currently  in  the  U.S.  Army's 
inventory. . 

1.4.2.  MethociolQ-gy- 

18.  Driving  a  vehicle  is  a  visually  intensive  task. 
Enhanced  resolution,  stereovision,  and  colour  are  anticipated 
to  provide  valuable  cues  for  optimizing  teleoperator 
performance.  Ideally,  we  would  like  to  compress  all  this 
visual  information  down  to  16  kilobits,  transmit  it  over  a 
single  data  channel  and  reconstruct  the  full  video  image  with 
no  loss  in  visual  quantity  or  quality  at  the  teleoperator's 
control  station  (see  Figure  8) .  This,  however,  is  not 
currently  within  the  state  of  the  art.  Therefore,  we  must 
identify  the  minimum  amount  of  visual  information  required 
for  remote  driving  and  find  ways  of  reducing  the  bandwidth  of 
the  video  transmission  that  preserve  or  synthesize  that 
visual  information  essential  to  the  performance  of  this 

task . . 

19.  Before  examining  the  influence  of  degraded  imagery 
on  remote  iriver  performance,  it  is  necessary  to  select  a 
lens  focal  length  for  the  video  camera  that  will  provide  the 
teleoperator  vision  forward  of  the  vehicle.  Although  shorter 
focal  lengths  will  increase  peripheral  vision,  resolution 
will  decrease  and  distortions  will  be  induced.  The  lens  focal 
length  will  also  effect  relative  distance  judgment  at  varying 
distances  from  the  vehicle.  As  focal  length  decreases, 
objects  appear  farther  away  than  they  are;  as  focal  length 
increases,  objects  appear  closer  than  they  are. 


20.  The  first  in  the  series  of  ROAD  RUNNER 
investigations  began  in  July  1990.  The  purpose  of  this  study 
was  to  select  a  lens  focal  length  for  the  fixed,  black  and 
white  video  camera  which  would  serve  as  a  baseline  during 
follow-on  investigations.  The  study  was  conducted  on  the 
indoor  test  course  using  the  Road  Runner  research  platform. 
During  this  investigation  remote  driver  performance  was 
•  measured  under  three  focal  length  or  FOV  conditions:  12  mm 

(29°),  6  mm  (55°)  and  3.5  mm  (94°).  Driver  performance  in  the 
on-board  mode  was  also  measured.  A  major  technology 
exposition  held  within  the  indoor  facility  and  subsequent 
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construction  of  laboratory  workspace  forced  postponement  of 
study  completion  until  this  coming  spring.  To  date,  9  of  the 
18  study  participants  have  completed  test.  Although  the  data 
collected  at  this  point  are  insufficient  to  perform  an 
analysis,  the  trends  appear  to  favor  a  focal  length  of  6  mm 
which  provides  a  55°  horizontal  FOV. 

21.  Follow-on  studies  will  seek  to  define  that  point 
where  reductions  in  video  quality  critically  impact  the 
soldier-operator's  ability  to  control  the  vehicle  under  field 
conditions.  These  studies  will  involve  a  systematic 
assessment  of  the  effects  of  reductions  in  the  number  of 
pixels,  gray  shades,  and  frame  rate  on  remote  driving 
performance.  An  initial  examination  of  the  influence  of 
colour  and  stereovision  will  also  be  conducted.  The 
information  obtained  during  these  investigations  will  assist 
in  the  selection  of  hybrid  image  compression  and  enhancement 
techniques  for  subsequent  assessment. 

1.4.3.  Image  Compression  and  Enhancement  .Options 

22 .  The  HEL  is  a  member  of  a  low  data  rate  working 
group  established  to  support  the  Robotic  Testbeds  program. 
This  group  meets  periodically  to  review  contractor  proposals 
and  the  status  of  efforts  in  the  development  and  assessment 
of  the  various  signal  processing  approaches.  As  a  group,  they 
maintain  current  knowledge  in  the  field  of  image  processing. 
They  become  intimately  familiar  with  those  approaches  that 
are  currently  being  developed  and  choose  which  techniques 
should  proceed  or  be  terminated. 

23.  Table  1  is  a  list  of  signal  processing  approach 
options.  These  approaches  fall  into  two  basic  categories: 
spatial  and  temporal.  During  initial  studies  conducted  by  the 
HEL,  the  individual  and  combined  effects  of  two  spatial 
techniques  (resolution  and  gray  scale  reduction)  and  one 
temporal  technique  (frame  rate)  will  be  assessed. 
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24.  Follow-on  studies  will  examine  the  influence  of 
colour.  Within  the  colour  domain,  red,  green  and  blue  may  be 
manipulated  to  achieve  the  effect  of  colour  at  less  cost  in 
bandwidth.  This  variable  set  may  be  transformed  into  hue, 
chroma  and  value  (see  Figure  9).  Only  those  colours  that  are 
important  in  that  domain  need  be  selected  and  transmitted. 

25.  Another  spatial  technique  planned  for  evaluation  is 
called  foveal-peripheral  or  foveal  window.  This  approach 
imitates  human  vision  creating  an  area  of  high  resolution  at 
the  center  of  the  image  surrounded  by  an  area  of  lower  or 
gradually  degrading  resolution  around  the  periphery.  The  high 
resolution  portion  of  the  image  may  be  maneuvered  to  a  point 
of  interest  in  the  display  by  either  an  eye  scanner 
(occulometer)  or  a  hand-controller. 


white  Yellow 


Figure  9.  Colour;  another  world 
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26.  Image  warping  is  a  geometrical  mapping  technique 
that  uses  information  such  as  camera  angle  and  vehicle  speed 
to  predict  how  the  information  in  the  picture  will  change 
over  time.  This  temporal  technique  could  be  used  to  fill  in 
missing  frames  and  possibly  minimize  the  effects  of  frame 
rate , reduction. 

27.  Not  all  processing  techniques  that  may  be  used  to 
reduce  bandwidth  will  effect  a  loss  in  image  quality.  "No 
loss"  techniques  include  frequency  domain  and  encoding.  There 
are  currently  two  "no  loss"  methods  of  encoding:  run  length 
encoding  and  the  "Huffman"  technique.  In  the  run  length 
encoding  technique,  the  value  of  a  pixel  and  the  number  of 
those  pixels  within  a  given  run  may  be  transmitted  instead  of 
the  individual  pixel.  The  Huffman  technique  is  a  coding 
schema  wherein  most  frequently  used  gray  shades  (or  colours) 
are  assigned  a  different  coding  with  fewer  bits  and  the  least 
frequently  used  gray  shades  assigned  the  full  8  bits.  Both  of 
these  techniques,  however,  are  time  consuming  and  may 
introduce  some  lag  in  the  transmitted  image. 

28.  Some  image  processing  techniques  will  work  more 
effectively  than  others  depending  on  the  type  of  knowledge  to 
be  compressed.  For  example,  an  image  containing  only  edges 
(high  frequency)  could  be  compressed  by  coding  only  the  edge 
knowledge.  However,  an  image  with  reduced  gray  shades  (low 
frequency)  would  use  an  entirely  different  approach  to 
achieve  maximum  compression.  These  different  pieces  of 
knowledge  may  be  compressed  individually  to  achieve  the 
optimum  effect  and  transmitted  at  lower  cost  in  bandwidth  by 
sending  them  separately  and  recombining  them  at  the  remote 
operator's  station. 

29.  These  various  processes  will  be  performed  by  the 
HEL  using  Versa  Modular  European  (VME)  based  boards  that  may 
be  pipelined  to  effect  the  desired  compression  technique  or 
enhance  the  image  artificially.  Most  of  this  hardware  is 


manufactured  by  DataCube.  The  software  package,  used  as  the 
real-time  operating  system,  is  called  Vx  Works,  developed  by 
Wind  River  System  Incorporated. 

1.  .5  SUMMARY 

30.  The  HEL,  in  partnership  with  the  USCSTA,  is 
developing  facilities  at  Aberdeen  Proving  Ground  to  support 
research  and  test  of  robotic  systems.  These  facilities 
include  an  indoor  and  outdoor  test  course  instrumented  for 
automatic  data  collection.  A  remotely  controlled  golf  cart  is 
among  the  tools  used  by  the  HEL  to  study  human  factors  design 
and  interface  issues  associated  with  teleoperation.  This 
platform,  along  with  a  teleoperated  HMMWV,  is  currently  being 
used  in  a  program  of  research  in  low  data  rate  driving.  This 
program  seeks  to  optimize  soldier-operator  performance  under 
tactical  conditions  requiring  reduced  bandwidth 
transmissions.  During  this  research,  the  influence  of  various 
image  compression  and  enhancement  techniques  on  remote  driver 
performance  will  be  measured.  In  April  1992,  the  results  of 
these  studies  will  be  demonstrated  during  low  data  rate 
driving  of  remotely  operated  ground  vehicle. 
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13.  Ketones/ Case. -i poors:  , 

Blackboard  -  Sensor  Planning  -  Sensor  Fusion  -  Perception  -  Robotics 
Artificial  Intelligence. 


14,  Abstract: 

Mobile  robots  which  have  to  move,  survey  and  achieve  interventions  in 
hostile  conditions  need  perceptual  functionnalities  that  allow  them 
to  intelligently  interpret  the  evolution  of  the  environment. 

Redundant  and  complementary  sensors  together  with  a  complete  library 
of  data  processing  are  all  necessary  to  build  reliable  and  safe  robotic 
systems.  The  hardware  and  software  resources  are  managed  by  an 
intelligent  perception  system  which  controls  all  the  choices  and 
paraoetrizations. 

The  follow  ing  paper  presents  the  GESPER  system  (GEStion  de  la  PERCEPTIO 
being  developed  at  the  LETI  laboratory  of  CEA.  This  system  is  based  on 
Artificial  Intelligence  concepts.  According  to  the  mission  assigned 
to  the  robot,  it  carries  out  the  perceptual  resources  control,  the 
management  of  the  perception  strategies  and  the  interpretation  of 
the  sensors  data. 
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1)  INTRODUCTI£>:  -  .  4.  ■ 

Although  many  works  were  devoted  to  path  planning  systems,  few  concerned 
perception  control  systems.  This  is  probably  due  to  the  fact  that  navigation  is  the  main 
problem  for  a  robot  Therefore,  there  are  many  algorithms  for  path  planning  and  testbeds  to 
validate  them.  But  the  perception  is  a  real  problem  which  has  to  be  approached 

The  first  question  one  can  answer  is :  why  does  a  robot  need  to  control  its  perception? 
The  essential  reason  is  the  need  for  adaptability  of  the  robot  A  robot  has  to  move  according 
to  the  constraints  of  the  environment  and  the  assigned  mission.  It  has  to  optimaly  schedule 
its  perceptual  resources  which  are  managed  by  an  intelligent  controller. 

1.1  Constraints  : 

In  general,  robots  move  in  complex  environments  that  can  be  unstructured,  unknown 
(robots  have  few  a  priori  knowledge  about  their  environment).  They  are  also  not  pre¬ 
equipped  which  excludes  the  use  of  s'ensors  based  on  the  recognition  of  active  beacons.  But 
the  most  important  characteristic  of  the  environment  is  its  changing  evolution.  Robots  have 
to  detect  any  changes  and  unexpected  events  like  moving  obstacles,  smoke,  changing  in  the 
lighting,  fog,...  and  to  react  intelligently. 

Because  of  these  constraints,  a  mobile  robot  cannot  be  pre-programmed.  It  must  be  a 
non-deterministic  reai-time  system  resolving  on  line  perception  resources  conflicts. 

1.2  Perceptual  resources  : 

The  perceptual  resources  of  a  robot  are  the  sensors  and  the  associated  data  processing. 
The  sensors  acquire  information  in  the  environment  and  the  associated  data  processing  help 
the  robot  to  interpret  this  information.The  number  and  the  type  of  sensors  and 
funcdonnalities  depend  on  the  missions  assigned  to  the  robot. 

As  any  single  sensor  is  imperfect,  a  reliable  perception  system  must  include  a  set  of 
redundant  and  comp lemen tan'  sensors  and  must  be  able  to  handle  errors  and  uncertainties. 

The  perception  control  system  analyzes  the  mission  and  answers  the  four  questions  : 
what  is  to  be  perceived,  where,  when  and  how  must  perceive  the  robot  in  order  to  complete 
this  mission.  Therefore,  the  mission  has  to  be  divided  into  perceptual  tasks.  The  perceptual 
tasks  are  then  planned  and  put  into  execution  by  the  system.  This  is  done  by  an  intelligent 
controller. 

1,3  Intelligent  perception  control  : 

As  the  sensors  are  redundant  and  complementary,  each  perceptual  task  may  be 
performed  using  one  of  several  perceptual  resources  and  data  processing.  So  the  first 
functionnality  of  an  intelligent  control  is  the  choice  and  the  parametrization  of  the  sensors. 
The  sensors  should  be  perfectly  characterized  as  the  perception  system  uses  the  sensors 
models  to  manage  the  perception. 
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Bat  it  is  not  the  only  funcdonnality  of  the  perception  system.  The  interpretation 
process  is  another  important  funcdonnality.  Having  acquired  data  from  the  sensors,  the 
system  has  to  use  this  data  to  localize  the  robot  and  to  build  representations  of  the 
environment. 

The  localization  process  supplies  the  absolute  *;r  relative  position  of  the  robot. 

As  for  representations,  there  is  not  a  single  representation  of  the  environment :  it  may 
be  fine  or  coarse,  on  all  the  surroundings  of  the  robot  or  only  on  small  areas  of  interest..". 
The  type  of  representation  depends  on  the  levels  of  description  needed  by  the  decision  level 
of  the  robot.  The  more  complex  the  decisions  the  robot  has  to  take,  the  more  complete  the 
representation  must  be.  Three  hierarchical  levels  are  usually  considered  :  symbolic, 
topologic  and  geometric. 


In  fact,  the  characteristics  of  a  good  perception  system  controller  are  : 

-  reliability  :  the  mission  must  always  be  completed, 

-  adaptability  :  the  controller  makes  the  robot  react  to  any  event, 

-  modularity  :  it  must  be  as  modular  as  possible  to  add  or  exclude  sensors  and 
data  processing, 

-  real-time. 


in  order  to  manage  the  complexity  of  the  different  alternatives : 

-  choice  and  paramerization  of  the  sensors, 

-  choice  and  parametrization  of  data  processing, 

-  choice  of  the  multisensory  fusion  method, 

-  choice  of  the  perception  strategies. 
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And  it  is  important  to  note  that  the  perception  system  is  closely  linked  to  the  decision 
system  and  the  term  perception  control  must  include  perception  and  decision  processes. 

In  this  section,  the  aim  and  functionnalities  of  a  perception  control  system  were 
defined.  The  following  sections  deal  with  the  different  architectures  suitable  for  a  perception 
system.  The  architecture  of  our  system  is  then  presented. 
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2)  PERCEPTION.  SYSTEM  ARCHITECTURES 

As  the  robot  has  to  work  and  reason  on  imprecise  and  evolving  information,  a  control 
system  needs  to  be  a  knowledge-based  system  mixing  symbolic  and  numerical 
representation  of  the  environment. 


Several  artificial  intelligence  architectures  have  already  been  put  forward  to  organize 
and  structure  the  intelligent  controller  of  a  mobile  robot.  These  architectures  belong  to  two 
main  groups :  the  behavioural  ones  (Brooks...)  and  the  functional  one  (Meystel,  Albus ...). 
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2.1  The  functional  architecture  : 

The  functional  architecture  seems  more  natural  because  it  divides  the  problem  into 
sub-problems,  data  flows  between  modules  are  simple  and  solutions  are  not  detailed  too 
early  during  planning.  But  it  is  a  rigid  system  and  it  is  difficult  to  integrate  new  modules  in 
this  type  of  architecture.  In  most  cases,  this  architecture  produces  very  complex  systems 
that  provide  global  solutions  even  for  a  simple  problem. 
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2.2  The  behavioural  architecture  : 

In  a  behavioural  architecture,  modules  are  independant  and  asynchronous.  The 
structure  is  flexible  but  the  difficulty  is  to  make  the  different  modules  communicate  and  it  is 
more  and  more  difficult  as  the  number  of  modules  increases.  And  the  control  of  the  whole 
system  is  the  main  problem. 

So,  the  tendancy  is  to  mix  those  two  approaches.  This  is  true  for  the  global  control 
system  of  a  robot  but  also  for  the  perception  control  system  which  in  fact  is  integrated  in  the 
former.  The  approaches  based  on  blackboards  are  particularly  interesting  as  a  robot  is  a  set 
of  agents  which  must  be  controlled  by  a  supervisor.  In  the  coming  section,  we  are  going  to 
detail  the  blackboard  approach. 

3)  BLACKBOARD  ARCHITECTURES 

Blackboards  are  distributed  structures  which  have  the  advantages  of  the  behavioural 
architecture  (modularity,  flexibility,  asynchronism)  but  the  communications  are  much  easier 
:  they  are  made  through  a  hierarchical  data  base.  They  also  take  advandtage  of  the  natural 
hierarchy  of  the  system. 


Blackboard  architecture 


Blackboard  is  the  name  of  the  architecture  and  the  name  of  the  hierarchical  data  base. 
Knowledge  Sources  (KS)  are  independant  processes  that  place  and  retrieve  data  from  the 
blackboard.  They  communicate  and  interact  only  via  the  blackboard  data  base.  The 
execution  is  so  event  driven  and  the  blackboard  architecture  contains  synchronisation 
primitives  that  determine  when  knowledge  sources  are  to  be  executed. 

We  have  developed  an  intelligent  perception  controller  based  on  this  blackboard 
architecture  and  we  are  going  to  give  details  about  our  implementation. 
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4)  THE  OESPER  SYSTEM 

GESPER  (acronym  for  GEStion  de  la  PERception)  is  the  name  of  our  perception 
control  system.  It  is  connected  to  the  navigation  module  :  the  path  followed  by  the  robot  is 
planned  thanks  to  the  representation  of  the  environment  supplied  by  GESPER  and  GESPER 
needs  the  path  to  plan  its  perception. 

GESPER  receives  one  or  several  missions  from  an  operator  with  execution 
constraints  (quickness,  safety ...). 


GESPER  is  implemented  on  TRAM,  a  blackboard  tool  developed  in  LETI  (in  C 
language).  The  events  deduced  by  GESPER  are  recorded  on  two  blackboards 
corresponding  to  the  two  main  processes  :  the  perception  planning  blackboard  for  the 
planning  process  and  the  perception, imeqiretarion  blackboard  for  the  data  interpretation 
process. 

The  top-down  perception  planning  process  analyzes  the  mission  and  provides 
perception  plans  for  the  sensor  activation  description  (which  sensor  has  to  be  used,  at  what 
time  and  place  and  how  it  must  be  parametrized),  the  parametrization  of  data  processing  and 
die  choice  of  the  wav  to  fuse  sensory  data.  The  perception  planning  module  also  carries  out 
the  execution  control"  of  the  generated  plans.  It  reacts  dynamically  to  any  unexpected  change 
in  the  environment  or  in  the  robot  (operator  intervention,  sensor  failure...).  It  replans  if 
possible  or  sends  information  to  the  navigator  if  not. 
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The  perception  actions  are  sent  to  the  actuators  and  GESPER  gets  back  raw  data  from 
sensors.  The  bottom-up  data  interpretation  process  makes  them  interpreted  by  tne  relevant 
processing  and  selects  the  right  fusion  algorithm  according  to  the  perceptual  goals. 

The  two  processes  communicate  with  an  a  priori  database  (read/update)  and  a 
knowledge  base  (read).The  a  priori  database  records  all  the  knowledge  about  the 
environment  (maps  for  example).  It  is  matched  with  the  information  extracted  by  the  data 
interpretation  module.  It  includes  beacons  and  landmarks  positions  and  models  if  existing. 
The  knowledge  base  records  all  knowledge  required  for  the  choice  and  the  parametrization 
of  sensors  and  data  processing.  For  example,  sensor  modelling  consists  in  the  description 
of  characteristics  such  as  orientation,  field  of  view,  ...  Fusion  and  data  processing 
algorithms  are  described  in  terms  of  :  CPU  time,  type  of  information  on  which  drey  can  be 
applied,  accuracy  ...  Perception  strategies  are  also  part  of  the  knowledge  base  (best 
position  to  perceive  landmarks  for  example) 


Q  operator) 


mission 


The  GESPER  architecture. 
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GESPER  has  got  four  perceptual  resources  : 

-  a  set  of  three  wide  field  cameras, 

-  a  time-of-flight  range  finder  (2D  scanning), 

-  a  camera  associated  with  a  structured  strip  light  (which  can  be  oriented) 

-  a  localization  camera. 

It  also  takes  into  account  odometer  data  to  estimate  the  robot  position. 

The  data  processing  are  related  to  the  perceptual  resources.  Data  processing  are  high 
level  algorithms  which  chain  pre-processing,  primitives  extraction,  primitives 
interpretation...  They  are  used  for  free-space  detection,  map  building  (2D  and  3D), 
landmark  recognition  and  localization. 

5)  CONCLUSION 

In  this  paper,  we  have  explained  the  need  of  an  intelligent  perception  control,  we  have 
described  the  different  existing  approaches  and  the  perception  control  system  GESPER 
based  on  the  blackboard  approach. 

GESPER  will  be  used  for  an  indoor  robot  equipped  with  a  stereo-vision  system,  a 
telemeter  and  two  cameras.  It  is  of  great  interest  for  all  robotic  applications  as  it  is  capable  to 
autonomously  plan,  trigger  acquisitions,  integrate  and  interprete  multisensory  data. 

GESPER  provides  a  generic  development  architecture  :  new  sensors  and/or  new  data 
processing  can  be  easily  integrated  depending  on  the  application.  It  is  a  modular,  flexible 
and  asynchronous  architecture  able  to  cope  with  either  outdoor  or  indoor  environments. 
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Cooperative  Control 

By:  xjavid  W.  Payton  and  Charles  P.  Dolan 
Hughes  Research  Labs,  301 1  Malibu  Canyon  Rd.,  Malibu,  CA  90265 

1.  INTRODUCTION 

We  describe  in  this  paper  an  approach  that  allows  the  activities  of  multiple  robotic  vehicles  to 
be  controlled  in  such  a  way  that  each  vehicle  is  capable  of  responding  to  unanticipated 
circumstances  in  a  highly  opportunistic  manner  while  still  adhering  to  the  constraints  of  high 
level  plans.  Our  experience  has  shown  that  conventional  approaches  to  plan  representation  and 
plan  execution  have  serious  weaknesses  when  applied  to  any  system  that  must  operate  in  a 
dynamic  environment.  Autonomous  vehicles  cannot  be  controlled  in  a  purely  top-down 
manner  if  they  are  to  be  truly  responsive  and  opportunistic.  Using  a  bottom-up  approach,  we 
show  how  each  vehicle  may  be  provided  with  the  competence  needed  to  perform  the  basic 
functions  required  for  obstacle  avoidance,  pursuit  and  evasion,  and  coordination  with  other 
vehicles.  High  level  plans  are  then  employed  as  an  additional  source  of  sensory  input  to 
improve  the  performance  of  those  capabilities  already  in  place.  By  developing  representations 
that  allow  multi-agent  plans  to  be  expressed  in  terms  of  constraints  and  advice  rather  than  as 
explicit  instructions,  we  show  how  a  single  plan  may  be  used  to  guide  the  activities  of  many 
vehicles.  Many  aspects  of  multiple  vehicle  control  are  thus  greatly  simplified  using  this 
methodology. 

Previous  work  has  addressed  the  problems  of  coordinating  multiple  cooperating  agents  both  in 
a  purely  top-down  plan-based  manner  and  a  purely  bottom-up  behavior-based  manner.  Plan- 
based  approaches  are  generally  concerned  with  generating  a  plan  that  will  completely  specify 
the  actions  of  each  agent.  In  behavior-based  approaches,  coordination  is  obtained  as  an 
emergent  property  of  many  simple  and  local  rules  of  interaction.  Our  work  seeks  to  establish  a 
common  ground  between  these  two  extremes. 

In  purely  plan-based  approaches,  all  actions  and  interactions  are  planned  in  advance.  The  plan 
is  then  a  script  for  action  that  all  cooperating  agents  must  follow  faithfully.  Assuming  that  the 
agents  can  each  execute  their  plan  correctly,  then  an  overall  coherent  activity  will  take  place. 
Plans  are  usually  expressed  either  logically  or  geometrically.  Using  a  logical  description  of  the 
environment  and  the  ways  the  environment  can  change  when  actions  are  performed,  logical 
proofs  can  be  used  to  determine  a  sequence  of  actions  for  each  agent  that  will  result  in  an 
intended  goal  state  [1],  In  more  spatially  oriented  multi-agent  path-finding  problems, 
geometric  reasoning  and  search  techniques  can  be  used  to  provide  reasonable  path  descriptions 
[2].  In  both  cases,  the  p!'5n<s  ^re  assumed  to  provide  a  complete  description  of  all  required 
actions.  Uncertainties  in  the  environment  must  be  accounted  for  by  contingency  actions  within 
the  plan.  This  assumption  leads  to  brittleness  should  the  original  problem  description  fail  to 
capture  all  the  uncertainties  of  the  real-world  environment. 

In  behavior-based  approaches,  intelligent  action  is  a  manifestation  of  many  simple  processes 
operating  concurrently  and  coordinated  through  the  context  of  their  environment.  Work  by 
Brooks,  for  example,  has  resulted  in  a  variety  of  robots  that  can  perform  complex  tasks 
without  using  plans  at  all  f3).  These  systems  exhibit  a  great  deal  of  robustness  because  their 
actions  are  in  direct  response  to  immediate  sensory  input.  Control  decisions  and 
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representations  are  determined  relative  to  each  specific  task,  so  that  environmental  uncertainties 
can  be  managed  on  a  task  by  task  basis. 

A  behavior-based  approach  can  produce  coordinated  multi-agent  actions  in  the  same  way  it 
produces  meaningful  action  in  a  single  agent.  Franklin,  for  example,  has  shown  in  simulation 
how  three  predators  can  be  made  to  chase,  encircle,  and  close  in  on  a  prey  using  only  local 
information  [4].  Each  agent  in  this  simulation  knows  only  its  relationship  to  other  agents 
within  a  very  limited  range.  A  very  simple  set  of  decision  rules  is  used  by  each  agent  to  evoke 
actions  in  response  to  the  actions  of  others  nearby.  Surprisingly,  while  the  resulting  actions 
appear  very  well  orchestrated,  no  explicit  communication  between  agents  is  required.  More 
recently,  Miller  [5]  has  suggested  a  number  of  simple  local  strategies  that  could  be  used  by  a 
team  of  agents  for  exploration  and  sample  recovery  on  Mars.  Again,  no  explicit  plan  is  used, 
and  only  minimal  communication  is  required.  Much  of  the  work  in  Artificial  Life  [6]  also 
exemplifies  this  approach.  By  giving  each  agent  the  same  set  of  procedures  for  how  to  behave 
in  response  to  the  actions  of  others,  a  variety  of  interesting  and  useful  group  behaviors  can 
emerge  [7]. 

Our  intent  is  to  take  advantage  of  the  robustness  of  a  behavior-based  approach  while  also 
providing  a  plan-based  means  for  controlling  a  team  of  agents.  Given  a  team  of  agents  with 
built-in  competences  for  various  forms  of  cooperation,  we  wish  to  be  able  to  direct  the  actions 
of  this  team  such  that  long-range  objectives  may  be  accomplished.  Explicit  control  of  each 
vehicle  will  not  suffice  in  this  case,  because  we  would  lose  the  robustness  inherent  in  the 
behavior-based  control  strategy.  Instead,  we  apply  the  notion  of  using  plans  as  resources  for 
action  [8].  As  resources,  plans  serve  as  sources  of  information  and  advice  to  agents  that  are 
already  fairly  competent  at  dealing  with  the  immediate  concerns  of  their  environment. 
Consequently,  plans  are  used  to  bias  the  natural  actions  of  agents  so  that  they  conform  more 
closely  to  achieving  some  global  objective. 

2.  INTERNALIZED  PLANS 

In  the  case  of  a  single  agent,  we  have  found  that  classical  plan  representations  serve  poorly  as 
a  resource  for  action  because  they  lack  much  of  the  information  needed  to  allow  flexible  and 
opportunistic  response  to  unanticipated  events.  We  have  developed  tire  concept  of  internalized 
plans  to  address  this  problem  [9].  An  internalized  plan  is  fundamentally  a  plan  representation 
that  allows  the  information  obtained  during  look-ahead  and  search  to  be  used  directly  within 
the  decision-making  processes  of  a  behavior-based  agent.  The  plan  is  not  intended  to  specify 
all  actions  of  the  agent.  Rather,  the  agent  is  expected  to  already  have  the  built-in  competence 
needed  to  deal  with  common  situations.  The  plan  is  merely  used  as  a  supplementaiy  source  of 
information  within  the  agent’s  decision-making  processes. 

To  clarify  how  internalized  plans  arc  used,  we  must  first  clarify  the  nature  of  an  agent’s  built- 
in  competence.  In  our  approach,  all  responses  of  an  agent  to  its  environment  are  controlled 
through  a  set  of  loosely  coupled  control  processes  we  call  reflexive  behaviors  [10].  These 
behaviors  each  consider  only  particular  aspects  of  the  environment  through  specialized  sensory 
processing  units  we  call  virtual  sensors.  Each  virtual  sensor  and  behavior  pair  serves  as  a 
direct  sensory-action  control  loop.  In  Figure  1,  shows  a  simplified  example  of  behaviors  for 
obstacle  avoidance  combined  with  behaviors  for  road  following  and  navigation.  Despite  the 
fact  that  these  behaviors  do  not  share  a  single  coherent  representation  of  the  world,  the 
appropriate  fusion  of  behavior  commands  [11]  allows  colicrcnt  actions  to  result. 
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O  ■  reflexive  behavior 


Figure  1:  Reflexive  behaviors  and  virtual  sensors  provide  the  basis  for  an  agent’s  fundamental 

competences. 

The  importance  of  a  behavior-based  approach  is  that  it  allows  us  to  construct  an  intelligent 
agent  in  terms  of  progressive  levels  of  competence  [12].  Using  behaviors,  fundamental  levels 
of  competence  such  as  basic  movement  and  survival  skills  are  implemented  first,  with  more 
sophisticated  skills  added  incrementally  as  needed.  Our  agents,  therefore,  do  not  require  plans 
to  deal  with  most  issues  in  their  environment.  All  capabilities  for  movement  and  avoidance  of 
sensed  obstacles  and  threats  are  implemented  directly  within  the  basic  control  processes  of  the 
agent 

At  some  point,  even  the  most  competent  agent  can  benefit  from  a-priori  knowledge  and 
planning.  Behavior-based  responses  based  solely  on  sensed  data  have  the  potential  perform 
immediate  actions  that  could  lead  to  long-term  difficulties.  When  choosing  between  two 
candidate  paths,  some  knowledge  about  where  each  path  will  lead  is  usually  quite  helpful. 
However,  if  this  kind  of  knowledge  is  to  be  used  effectively  by  a  competent  agent,  then  it 
cannot  be  used  to  control  the  agent  directly.  Plans  must  serve  as  sources  of  information  to  the 
agent,  having  no  more  influence  over  the  agent’s  actions  than  the  sensory  inputs. 

Internalized  plans  provide  a  means  to  bias  the  actions  of  a  competent  agent.  As  shown  in 
Figure  2,  a  gradient  field  is  one  foim  of  internalized  plan.  Note  that  the  plan  does  not  indicate 
an  explicit  path  to  the  goal.  Instead,  the  plan  provides  advice  about  desirable  headings  to  get  to 
the  goal.  Given  any  vehicle  location,  the  plan  will  provide  a  suggested  heading.  The 
behavior-based  control  system  has  the  responsibility  to  use  this  advice  in  a  way  that  is 
appropriate  for  the  current  situation.  The  vehicle  can  never  be  expected  to  follow  the  advice  of 
the  plan  exactly  because  unknown  obstacles  will  inevitably  appear  in  its  path.  While  the 
obstacle  avoidance  behaviors  have  the  built-in  competence  to  handle  such  circumstances,  they 
can  do  a  better  job  overall  if  they  also  take  the  advice  of  the  plan  into  account 
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Figure  2:  An  Internalized  Plan  for  a  Single  Vehicle  Route 


Internalized  plans  are  generated  with  search  and  are  used  by  indexing  into  the  results  of  that 
search  at  run  time.  In  the  example  above,  the  search  was  performed  as  a  shortest  path 
computation  in  a  map  [13],  The  state-space  for  this  search  is  simply  vehicle  locations. 
Transitions  between  states  are  determined  from  estimated  costs  of  moving  between 
neighboring  locations.  The  resulting  gradient  field  associates  with  each  location  state,  a 
pointer  to  the  best  subsequent  state.  In  order  to  use  the  internalized  plan,  the  vehicle’s  location 
relative  to  the  map  must  be  known.  The  current  location  becomes  an  index  into  the  map,  with 
the  resulting  advice  being  a  suggested  heading. 


An  important  issue  in  the  design  of  internalized  plans  is  the  sensitivity  of  the  indexing 
functions.  In  the  case  of  gradient  fields,  small  errors  in  position  usually  have  only  a  small 
effect  on  the  resulting  advice  from  the  plan.  Only  when  significant  discontinuities  in  tire  terrain 
exist  will  the  plan  advice  be  highly  sensitive  to  position  error  [14].  Therefore,  estimated 
position  is  usually  a  robust  function  for  indexing  into  a  gradient  field.  In  evaluating 
internalized  plan  representations  for  multi-vehicle  control,  the  robustness  of  the  indexing 
function  will  be  a  major  concern. 


In  effect,  internalized  plans  are  used  as  supplementary  sensory  inputs  to  the  behavior-based 
agent.  The  gradient  field,  for  example,  can  be  thought  of  as  a  phantom  compass  that  always 
gives  a  general  idea  of  the  right  way  to  go.  The  behaviors  that  use  this  information  operate  in 
the  same  way  that  sensor-based  obstacle  .r  oidance  behaviors  perform. 

3.  EMERGENT  COOPERATION 


The  concept  of  internalized  plans  can  be  applied  directly  to  a  multi-agent  domain.  Using  a 
team  of  behavior -based  agents,  a  single  plan  can  be  used  to  guide  the  actions  of  the  whole 
team.  Again,  each  agent  must  have  appropriate  competences  for  both  individual  and  group 
coordination.  Given  these  competences,  the  internalized  plan  need  only  provide  biases  to  each 
agent’s  actions  to  keep  the  team  properly  coordinated  toward  achieving  a  desired  objective. 
Because  internalized  plans  can  associate  a  bias  with  each  possible  situation,  multiple  agents 
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using  the  same  internalized  plan  can  each  receive  unique  information  relevant  to  their  own 
particular  situation  or  state. 

Many  fundamental  competences  required  for  cooperative  interaction  such  as  moving  in  groups 
and  staying  in  formation  can  be  supported  through  an  appropriate  set  of  behaviors  that  respond 
only  to  the  local  relationships  between  a  vehicle  and  its  neighbors.  By  designing  agents  to 
specialize  their  actions  according  to  their  local  interactions  with  other  members  of  their  team, 
they  will  be  able  to  automaticrily  adapt  to  the  loss  of  team  members,  and  will  be  able  to  assume 
roles  that  were  previously  handled  by  others.  Internalized  plans  may  then  be  used  to  guide 
these  competent  teams  toward  achieving  long-range  objectives. 

Figure  3  shows  an  example  of  how  the  combination  of  internalized  plans  and  behaviors  for 
local  interaction  can  be  used  to  achieve  mission  goals.  In  this  example,  it  is  desired  that  three 
vehicles  reach  a  specified  ridge  while  maintaining  a  linear  formation.  The  internalized  plan  for 
this  problem  is  indicated  by  the  small  arrows.  Note  that  the  plan  specifies  the  best  direction  to 
head  from  any  location  in  order  to  reach  the  ridge,  and  therefore  is  applicable  to  each  agent. 
Meanwhile,  each  vehicle  is  given  behaviors  that  try  to  maintain  a  constant  distance  between 
them.  Because  the  internalized  plan  is  used  as  advice  to  these  behaviors,  the  overall  actions 
taken  by  each  vehicle  will  be  determined  as  a  combination  of  these  influences.  Therefore,  the 
vehicles  will  approach  the  ridge,  but  if  one  is  diverted  to  avoid  an  obstacle,  the  others  will 
adjust  their  relative  positions  to  maintain  proper  spacing. 


Figure  3:  An  internalized  plan  to  guide  three  vehicles  to  a  ridge. 

By  providing  each  agent  with  an  appropriate  level  of  competence,  we  are  free  to  represent  plan 
information  in  forms  that  can  augment  the  performance  of  each  agent  and  of  the  group  as  a 
whole.  The  behaviors  provide  competence  to  each  individual  agent,  defining  local  interactions 
between  agents  in  such  a  way  as  to  establish  and  maintain  tactical  formations  within  groups  of 
agents  in  a  robust  distributed  fashion.  Internalized  plans  may  then  be  used  to  guide  the  actions 
of  these  agents  toward  user  defined  goals,  thereby  achieving  a  balance  between  local  autonomy 
and  supervised  control. 
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4.  MARKERS  AS  SENSORY  INPUT 

In  complex  multi-agent  domains,  internalized  plans  will  need  to  provide  far  more  information 
than  just  a  desired  heading.  The  relationships  between  a  vehicle  and  its  neighbors  as  well  as 
between  a  vehicle  and  the  surrounding  terrain  may  all  provide  relevant  influences  on  vehicle 
actions.  Because  of  the  potential  complexity  of  these  relationships,  plans  may  be  needed  to 
provide  advice  regarding  which  aspects  are  most  relevant  in  the  current  situation.  Because  we 
desire  this  advice  to  be  expressed  in  a  form  that  is  equivalent  to  other  sensory  input,  we  seek  a 
common  representation  that  is  suitable  for  both  real  sensory  input  and  plan  advice. 

We  have  found  that  the  concept  of  markers  as  proposed  by  Agre  and  Chapman  [15]  [16] 
provides  a  powerful  and  flexible  representation  that  is  ideally  suited  for  our  behavior-based 
architecture.  Markers  may  be  used  within  a  retinocentric  reference  frame  to  keep  track  of 
important  features  in  the  environment.  As  shown  in  Figure  4,  a  marker  may  be  associated 
with  visible  objects  such  as  neighboring  vehicles  and  critical  points  along  ridge  lines.  As 
visible  vehicles  move  relative  to  the  observer,  the  observer’s  perception  system  will 
automatically  keep  the  markers  co-located  with  the  observed  vehicles.  Over  time,  each  marker 
win  acquire  information  about  its  associated  object’s  range  and  estimated  motion. 


VISUAL  SCENE 


MARKERS 


Figure  4:  Markers  track  features  within  the  visual  scene. 

In  our  architecture,  we  make  a  distinction  between  markers  that  are  associated  with  objects  in 
the  visual  scene,  and  markers  that  may  be  moved  about  between  objects  to  identify  causal 
relationships  between  these  objects  and  the  observer.  The  first  type,  we  call  sensory  markers. 
The  second,  we  call  deictic  markers. 

Sensory  markers  are  produced  directly  from  our  virtual  sensors.  They  relate  to  aspects  of  the 
environment  that  are  directly  sensed  and  that  can  relate  directly  to  behavior  actions.  Sensory 
markers  track  various  aspects  in  the  environment.  For  example,  a  sensory  marker  may  be 
associated  with  each  observed  vehicle.  These  markers  will  track  the  observed  vehicles  in  the 
visual  scene.  Upon  initial  observation  of  another  vehicle,  a  marker  will  be  established  and 
perception  resources  will  be  devoted  to  tracking  that  vehicle  within  the  visual  scene.  The 
marker  will  follow  that  track.  Should  a  range  estimate  are  obtained,  the  associated  marker  will 
acquire  this  range  data  as  part  of  its  state. 

Deictic  markers  are  markers  that  may  be  moved  from  one  sensory  marker  to  another.  They 
represent  causal  relationships  between  the  agent  and  aspects  of  the  observed  environment. 
Once  associated  with  a  sensory  marker,  they  will  move  in  correspondence  with  the  sensory 
marker.  Deictic  markers  provide  a  powerful  mechanism  for  focusing  the  attention  on  relevant 
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aspects  of  the  environment.  For  example,  sensory  markers  may  identify  several  nearby 
vehicles.  Deictic  markers  can  then  be  used  to  identify  relationships  such  as  “THE-VEHICLE- 
THAT-SHOULD-STAY-TO-MY-LEFT “THE-VEHICLE-THAT-SHOUI  U-STAY-TO- 
MY-RIGHT,”  and  “THE-VEHICLE-i-AM-FOLLOWING.”  Chapman  has  described  how  the 
manipulation  of  markers  such  as  these  can  simplify  reasoning  processes  by  focusing  attention 
only  on  relevant  aspects  of  the  environment  [17].  ’  Vl  • 

5.  INDEXING  STATE  INTO  ADVICE 

Markers  provide  an  ideal  means  for  representing  the  advice  from  internalized  plans.  If  an 
internalized  plan  can  influence  marker  relationships  as  a  function  of  vehicle  state,  then  it  will 
have  a  direct  influence  on  the  vehicle  behaviors.  In  this  way,  the  plan  will  truly  be  treated  on 
an  equivalent  basis  with  real  sensory  input.  To  understand  how  plans  should  be  used  to 
manipulate  markers,  we  must  first  examine  the  kinds  of  information  that  may  be  needed  from 
plans.  • 

In  our  previous  examples,  sensory  markers  are  associated  with  objects  in  the  perceptual  field 
of  a  vehicle,  and  deictic  markers  are  moved  into  correspondence  with  sensory  markers  to  effect 
behavior.  However,  vehicle  behavior  often  should  be  contingent  on  data  that  is  not  present  in 
the  perceptual  field.  For  example,  in  Figure  5a,  we  see  that  there  is  a  vehicle  hidden  behind  a 
hill.  Normally,  this  hidden  vehicle  would  not  be  known  to  the  observing  vehicle.  However,  a 
variety  of  features  of  the  current  situation  might  provide  just  cause  to  suspect  the  presence  of 
the  hidden  vehicle.  Observations  such  as  tracks  in  the  dirt,  or  the  actions  of  other  nearby 
vehicles  may  be  sufficient  cues  in  this  case.  Using  only  deictic  and  sensory  markers,  it  would 
not  be  possible  for  any  behaviors  to  respond  to  this  object.  We  therefore  require  a  means  for 
creating  new  markers  for  hypothesized  objects. 


VISUAL  SCENE 


MARKERS 


Figure  5:  A  scenario  requiring  hypothesized  markers 

We  call  the  markers  created  for  unobserved  objects  hypothesized  markers.  These  markers  will 
be  generated  from  an  internalized  plan.  In  essence,  the  plan  embodies  the  advice  that  says 
“when  you  see  three  vehicles  in  this  configuration,  watch  out  for  another  behind  the  hill.”  The 
advice,  however,  is  more  specific  to  the  situation  than  this,  in  that  it  identifies  the  approximate 
location  of  the  suspected  vehicle  with  a  marker.  Such  advice  could  be  obtained  from 
projection  and  search,  or  it  could  be  retained  from  past  experience.  Like  the  simple  gradient 
field,  the  current  state  has  a  significant  impact  on  the  advice  obtained.  In  this  case,  though,  the 
state  is  far  more  complex  than  a  mere  location.  The  state  is  represented  by  relationships 
between  all  sensory  and  deictic  markers. 
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Merely  hypothesizing  undetected  objects  is  not  sufficient.  We  also  require  a  means  to  establish 
the  causal  relationships  between  both  detected  and  undetected  objects.  Again,  the  current  state 
of  the  vehicle  relative  to  its  surroundings  should  provide  information  relevant  to  establishing 
these  relationships.  This  state  will  include  past  reports  from  other  team  members,  relative 
positions  of  other  vehicles,  and  characteristics  of  the  local  terrain..  The  advice  that  results  will 
involve  the  manipulation  of  deictic  markers. 

As  an  example,  Figure  5b  shows  recognition  of  a  potential  ambush.  The  agent  viewing  this 
.scene  does  not  really  know  whether  or  not  there  is  a  vehicle  hiding  behind  the  hill.  Its 
internalized  plan,  however,  has  identified  this  possibility  by  creating  a  hypothesized  marker 
behind  the  hill.  The  internalized  plan  has  then  recognized  the  possibility  that  such  a  vehicle 
could  play  the  role  of  an  ambushing  vehicle.  It  therefore  has  placed  the  deictic  marker  “D”  on 
top  of  the  hypothesized  marker.  The  specific  deictic  marker  used  might  be  an  “AMBUSHING 
VEHICLE”  marker,  thereby  capturing  the  causal  relationship  between  the  hidden  vehicle  and 
the  observing  agent.  In  response  to  the  placement  of  this  marker,  vehicle  behaviors  can  now 
respond  with  caution  as  if  a  real  vehicle  were  known  to  be  in  that  location. 

Clearly,  the  problem  of  generating  suitable  markers  from  the  current  state  is  far  more  complex 
than  that  of  merely  following  a  gradient  field.  Sill,  the  basic  concept  remains  the  same.  Even 
the  gradient  field  can  be  thought  of  as  generating  a  simple  marker.  For  example,  we  could  use 
the  gradient  field  to  produce  a  hypothesized  goal  marker.  This  marker  would  move  as  the 
vehicle  moved  because  the  vehicle’s  state  would  constantly  be  changing.  The  resulting  action 
would  be  similar  to  that  of  a  horse  attracted  to  a  moving  carrot.  The  plan  moves  the  carrot,  but 
the  vehicle  behaviors  are  still  fundamentally  in  control. 

In  order  to  deal  with  complex  multi-agent  plans,  we  must  be  able  to  support  a  much  higher 
dimensionality  of  state.  In  the  gradient  field  example,  state  was  represented  merely  by  two 
numbers:  a  longitude  and  a  latitude.  Indexing  is  simple  in  this  case.  The  gradient  field  may 
be  represented  by  an  array,  and  indexing  is  performed  by  a  simple  array  look-up.  When 
sensory  and  deictic  markers  are  used  to  represent  the  current  state,  then  representation  and 
indexing  become  a  serious  problem.  In  multi-agent  plans  the  location  and  status  of  every  other 
agent  is  relevant.  Even  if  we  could  compute  such  a  large  dimensional  plan,  an  agent  does  not 
have  access  to  information  about  all  the  other  agents.  Therefore  we  need  a  plan  representation 
that  allows  us  to  retrieve  advice  given  partial  state  descriptions. 

The  technology  of  case-based  reasoning  (CBR)  gives  us  just  such  a  capability.  CBR  systems 
can  take  a  partial  state  description  and  efficiently  retrieve  a  closely  matching  full  state 
description  [18].  However,  states  predicted  during  planning  will  not  be  exactly  the  same  as 
states  actually  encountered  in  the  world.  Fortunately,  CBR  systems  can  also  take  a  full  state 
description  and  adapt  it  to  the  current  state  [19], [20].  Therefore  we  propose,  as  our 
internalized  plan  representation,  the  case-based  internalized  plan  (CBIP). 

Figure  6  shows  our  overall  architecture  using  the  CBIP.  Tire  retrieval  indices  of  the  CBIP  are 
all  of  the  markers,  including  hypothesized  markers,  that  are  examined  by  the  behaviors.  The 
output  of  the  CBIP  is  commands  or  advice  to  a  set  of  modules  we  call  virtual  operators. 
Virtual  operators  are  responsible  for  managing  all  hypothesized  and  deictic  markers.  They  me 
similar  to  the  virtual  sensors  except  their  input  data  consists  only  of  the  sensory  markers 
produced  from  the  virtual  sensors.  An  example  of  commands  sent  to  the  virtual  operators 
might  be  "Create  a  hypothesized  velucle  behind  die  hill  in  the  middle  of  the  perceptual  field." 
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SENSORY  INPUT  OUTPUT  COMMANDS 


Figure  6:  CBIP  Architecture 

Note  that  we  have  decomposed  the  problem  somewhat  differently  from  previous  marker  based 
approaches  [15], [17].  Agre,  for  example,  has  proposed  a  central  system  that  directs  a 
perceptual  system  [16].  The  functions  of  our  virtual  sensors  are  intended  to  be  similar  to 
Agre’s  “visual  routines.”  The  visual  routines,  however,  do  not  create  markers  in  Agre’s 
system.  The  only  markers  created  are  equivalent  to  what  we  have  called  deictic  markers.  We 
have  added  the  notion  of  sensory  markers  to  provide  the  needed  state  information  for  the 
CBIP.  Further,  we  have  added  the  notion  of  hypothesized  markers.  Because  deictic  and 
hypothesized  markers  are  managed  differently  from  sensory  markers,  we  have  split  the 
perceptual  system  into  two  distinct  components.  A  limitation  of  our  approach  that  remains  to 
be  addressed  is  the  inherent  inability  of  the  virtual  sensors  to  detect  all  objects  that  may  be 
relevant  to  the  behaviors. 

6.  GENERATING  PLANS 

Tie  generation  of  internalized  plans  places  constraints  on  the  type  of  planning  algorithms  that 
can  be  used.  First,  an  internalized  plan  encompasses  all  the  states  examined  during  planning. 
Therefore,  algorithms  that  are  postulated  on  re-using  memory,  such  as  depth-first  iterative 
deepening,  are  not  applicable. 

Second,  the  cunrent  state  of  the  agent  is  used  as  an  index  to  retrieve  advice  for  the  behaviors. 
Because  perceptual  cues  are  used  as  indices,  the  planner  must  search  in  the  space  of  operators 
and  states  that  correlate  well  with  actions  that  may  be  taken  by  the  agent  and  with  perceptual 
information  that  will  be  available  as  a  result  of  these  actions.  For  example,  some  planning 
systems  search  in  the  space  of  task  orderings,  incrementally  adding  constraints  to  the  ordering 
of  tasks.  Such  plans  are  difficult  to  use  as  internalized  plans  because  it  is  difficult  to  create  a 
function  that  indexes  world  states  into  task  ordering  constraints. 


Even  so,  the  planner  may  have  to  perform  extra  computations  during  planning  to  compute 
indices  from  marker  states  to  plan  states.  The  extra  work  may  be  required  because  operations 
on  markers  may  not  be  the  most  natural  search  space.  However,  if  the  state  space  corresponds 
well  to  the  environment  sensed  by  an  agent,  creating  marker  indices  should  not  be  problematic. 

The  last  constraint  is  the  choice  of  abstractions.  As  noted  before,  the  search  space  for  multi¬ 
agent  plans  has  many  dimensions,  and  therefore  abstraction  will  be  necessary.  However, 
because  we  have  the  constraint  of  being  able  to  index  the  plan  using  perceptual  cues,  we  would 
like  to  be  able  to  index  the  abstractions  in  the  same  manner.  A  good  example  of  an  applicable 
abstraction  approach  was  demonstrated  using  the  SOAR  program  [21].  In  this  very  general 
abstraction  approach,  abstractions  are  formed  by  dropping  parts  of  the  state  description.  In 
our  mutli-vehicle  example,  this  would  correspond  to  dropping  markers  from  the  state 
description,  or  dropping  marker  attributes. 

7.  CONCLUSION 

We  have  presented  an  approach  for  using  plans  as  resources  to  bias  the  actions  of  behaviors 
rather  than  to  force  a  particular  course  of  action.  As  a  result,  each  vehicle  is  given  sufficient 
flexibility  to  respond  to  unexpected  changes  in  its  environment.  In  formulating  this  concept  of 
using  plans  as  resources,  we  have  developed  a  methodology  for  representing  plans  as  visual 
input  to  vehicle  behaviors.  This  allows  plan  data  to  be  considered  on  an  equal  basis  with 
sensory  input  from  the  outside  world.  Plans  are  communicated  to  behaviors  through  the 
instantiation  of  markers  within  the  visual  field  of  die  robot.  These  markers  may  be  as  simple 
as  indicators  of  a  goal  direction  or  as  complex  as  indicators  that  track  potential  threats  and 
targets.  Markers  also  provide  a  unique  mechanism  for  human  operators  to  provide  advice  to  a 
semi-autonomous  system.  Because  markers  can  be  made  to  track  visual  features,  a  human 
operator  may  be  able  to  augment  the  performance  of  a  robotic  vehicle  by  manually  indicating 
important  features  that  were  not  found  automatically. 

8.  REFERENCES 

[1]  Georgeff,  M.P.  and  Lansky,  A.L.,  Reasoning  about  Actions  and  Plans:  Proceedings  of 
the  1986  Workshop,  Morgan  Kauffman,  1987. 

[2]  Mitchell,  J.S.B.  and  Wyntcrs,  E.L.,  "Optimal  Motion  of  Covisible  Points  Among 
Obstacles  in  the  Plane,"  School  of  Operations  Research  and  Industrial  Engineering, 
Cornell  University,  Technical  Report  896,  March,  1990. 

[3]  Brooks,  R.A.,  "Intelligence  Without  Representation,"  in  Preprints  of  the  Workshop  in 
Foundations  of  Artificial  Intelligence,  Endicott  House,  Dedham,  MA,  June,  1987. 

[4]  Franklin,  R.F.  and  Harmon,  L.A.,  "Elements  of  Cooperative  Behavior,"  ERIM,  Ann 
Arbor,  MI,  Internal  Research  and  Development  Report  655404-  1-F,  August,  1987. 

[5]  Miller,  D.P.,  "Multiple  Behavior-Controlled  Micro-Robots  for  Planetary  Surface 
Missions,"  in  IEEE  Systems  Man  and  Cybernetics  Conf,  Studio  City,  CA,  November, 
1990. 

[6]  Langton,  C.G.,  Artificial  Life ,  Addison-Wesley,  1989. 


C5.ll 


AC/243-TP/3 


[7]  Arai,  T.,  Ogata,  H.  and  Suzuki,  T.,  "Collision  Avoidance  Among  Multiple  Robots 
Using  Virtual  Impedance,"  in  IEEEIRSG  International  Workshop  on  Intelligent  Robots 
and  System,  Tsukuba,  Japan,  September,  4-6, 1989,  pp.  479-485. 

[8]  Suchman,  L.,  "Plans  and  Situated  Actions,"  Cambridge  University  Press,  1987. 

[9]  Payton,  D.W.,  "Internalized  Plans:  A  Representation  for  Action  Resources,"  Robotics 
and  Autonomous  Systems,  6(1),  1988,  pp.  89-103. 

[10]  Payton,  D.W.,  "An  Architecture  for  Reflexive  Autonomous  Vehicle  Control,"  in  IEEE 
International  Conference  on  Robotics  and  Automation,  San  Francisco,  CA,  April  7-10, 

1986,  pp.  1838-1845. 

[11]  Rosenblatt,  J.K.  and  Payton,  D.W.,  "A  Fine-grained  Alternative  to  the  Subsumption 
Architecture  for  Mobile  Robot  Control,"  in  International  Joint  Conference  on  Neural 
Networks,  Washington  D.C.,  June,  1989,  pp.  317-324. 

[12]  Brooks,  R.A.,  "A  Robust  Layered  Control  System  for  a  Mobile  Robot,"  IEEE  Journal 
of  Robotics  and  Automation,  RA-2(1),  1986. 

[13]  Mitchell,  J.S.B.,  Payton,  D.W.  and  Keirsey,  D.M.,  "Planning  and  Reasoning  for 
Autonomous  Vehicle  Control,"  International  Journal  for  Intelligent  Systems,  2, 1987. 

[14]  Payton,  D.W.,  "Circulation  Maps:  A  Resource  for  Identifying  Position  Accuracy 
Requirements,"  in  IEEEIRSJ  International  Workshop  on  Intelligent  Robots  and  Systems, 
Tsukuba,  Japan,  September  4-5, 1989,  pp.  448-455. 

[15]  Agre,  P.E.  and  Chapman,  D.,  "Pengi:  An  Implementation  of  a  Theory  of  Activity,"  in 
The  Sixth  National  Conference  on  Artificial  Intelligence,  Seattle,  Washington,  July, 

1987,  pp.  268-272. 

[16]  Agre,  P.E.,  "The  Dynamic  Structure  of  Everyday  Life,"  PhD  Thesis,  Massachusetts 
Institute  of  Technology,  1988. 

[17]  Chapman,  D.,  "Penguins  Can  Make  Cake,"  AI  Magazine,  10(4),  1989,  pp.  45-50. 

[18]  Goodman,  M.,  "CBR  in  Battle  Planning,"  in  Case-Based  Reasoning  Workshop,  1989, 
pp.  264-269. 

[19]  Kolodner,  J.L.,  "Extending  Problem  Solver  Capabilties  Through  Case-Based 
Inference,"  in  The  Fourth  International  Workshop  on  Machine  Learning,  1987,  pp.  167- 
178. 

[20]  Hammond,  K.J.,  "Case-Based  Planning:  A  Framework  for  Planning  from  Experience," 
Cognitive  Science,  14(3),  1990,  pp.  385-444. 

[21]  Unruh,  A.  and  Rosenbloom,  P.S.,  "Abstraction  in  Problem  Solving  and  Reasoning,"  in 
Proceedings  of  the  Eleventh  International  Joint  Conference  on  Artificial  Intelligence, 
Detroit,  Michigan,  August  20-25, 1989,  pp.  681-687. 


RENSEIGNEMENTS  BIBLIOGRAPHIQUES 


1.  Reference  du  destlnataire: 


2.  Autre  Reference: 


3.  Reference  d'orlgine: 


7.  Tltre  (NSC): 


4.  Classification  de  securite: 

SANS  CLASSIFICATION  ILLIMITEE 


5.  Date:  6.  Nbre  de  pages: 

Mars  1991  11 


Etude  de  concepts  de  robots  de  combat 


8.  Sourais  a: 

Seminaire  du  6RD:  Robotique  du  champ  de  bataille  (6-8  Mars  1991) 


9.  Auteur(s)/Editeur(s) : 

A.  Briand,  R.  Baton,  S.  Muller 


10.  Adresse  de  1 'auteur/editeur: 

CEA 
B.P.  6 

F-92265  Fontenay  Cedex 
France 


11.  Point  de  Contact  a  1'OTAN: 

Section  Recherche  pour  la  Defense 
Quartler  General  de  l'QTAN 
8-1110  Bruxelles 
Belgique 


12.  Distribution: 

La  distribution  de  ce  document  n'est  pas  soumise  aux  regies  de 
securite  de  1 'OTAN 


13.  Mots  cles/Descripteurs: 

Robotique,  Robots  telecommandes 


14.  Resume: 


A  partir  de  1 ' analyse  des  missions  de  l'armee  de  Terre  et  de 
l'etat  de  l'art  en  robotique,  teleoperations,  capteurs  et  armements, 
cette  etude  a  defini  des  concepts  de  robots  du  champ  de  bataille. 

Une  analyse  de  la  valeur  a  ensuite  ete  effectuee  sur  ces  concepts, 
et  des  tendances  a  moyen/long  temie  ont  ete  degsgees. 
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ETUDE  DE  CONCEPTS  DE  ROBOTS  DE  COMBAT 
1/  INTRODUCTION 

Le  CEA  et  GIAT  Industries  ont  decide  de  coordonner  leurs  efforts  dans  le  domaine  de  la  robo- 
dque  militaire.  Leur  accord  vise  a  developper  les  effets  de  synergie  pouvant  resuiter  de  l'expe- 
rience  de  recherche  du  CEA  et  de  la  pratique  de  conception  et  d'integration  de  materiels  militaries 
de  GIAT-Indusnies. 


Figure  1 


Le  CEA  ( Commissariat  a  iEnergie  Atomique)  et  GL\T-Industnes  ( Groupement  Industrie i  des  ' 
Armements  Terrestres )  ont  ete  retenus  ensemble  par  la  DAT  /  SEFT  ( Direction  des  Armements 
Terrestres,  Sections  d' Etudes  et  Fabrications  des  Telecommunications  )  pour  une  ''Etude  de 
concepts  de  robots  combat".  Cette  etude  de  dix-huit  mois  a  debute  mi  1989.  Son  objet  est 
I'etablissement  de  plans  d'actions  a  moven  terme  et  de  lignes  d'actions  a  long  terme  pour  le  de- 
veloppement  de  robots  de  combats  panrd  les  plus  utiles. 


PLAN  DE  L'ETUDE 


Figure  2 


Cette  etude  comprend  qoaore  phases,  Les  deux  premieres  d’entre  elles  ont  ete  effeetuees  en  pa- 
rallele;  elles  ont  pour  objet  de  determiner  les  aetes  militaries  robotisables  d’une  part,  et  de  reunri 
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la  documentation  utile  aux  phases  suivantes  d'autre  pan.  La  troisieme  phase  a  pour  objet  la 
conception  des  robots  sounds  a  la  selection  de  phase  4,  dont  1’objectif  final  est  l’elaboration  de 
plans  d’action  a  moyen  tenne  et  de  lignes  d’action  a  long  terme. 

21  ANALYSE  DES  MISSIONS  DE  L’ARMEE  DE  TERRE 

Cette  analyse  comprend  une  revue  generate  des  missions, 
leurs  decomposition  en  precedes,  puis  en  actes  elementaires  repedtifs. 

Les  activites  militaires  ont  ete  divisees  en  huit  domaines: 

ABC  (Anne  Blindee  Cavalerie) 

Infanterie 

Artillerie 

ALAT  (Aviation  Legere  de  l’Annee  de  Terre) 

Genie 

Train 

NBC 

Logistique. 

Dans  chacun  de  ces  domaines  ont  ete  etablies  des  listes  des  modes  d’action,  des  precedes  et  des 
actes  elementaires,  une  activite  d’un  niveau  etant  composee  d’une  ou  plusieurs  activites  du  ni¬ 
veau  inferieur,  exception  faite  pour  les  actes  elementaires  qui  constiment  le  niveau  le  plus  bas. 

Une  liste  de  criteres  a  ete  etablie:  limitations  dues  a  I’homme,  cout  en  personnel  (nombre  in- 
cluant  les  risques  et  qualite)  ou  en  materiel,  possibility  de  robotisation,  frequence  d’ occurrence 
de  1’ activite  consideree.  Parallelement  ont  ete  dennies  des  conditions  d’environnement  et  des  do- 
minantes  pour  permettre  de  determiner  les  situations  particulieres  pour  lesquelles  les  besoins  de 
robotisation  seraient  les  plus  importants. 

Chaque  acte  elementaire  a  ete  note  en  fonction  de  chacun  de  ces  criteres,  conditions  d’environne¬ 
ment  et  dominantes.  Le  resultat  de  ce  travail  a  ete  soumis  a  des  operatiotinels  qui  ont  fait  un  cer  ¬ 
tain  nombre  de  remarques  stir  les  activites  etudiees,  sur  certaines  activites  absentes  et  sur  les  no¬ 
tations  effectuees.Ces  remarques  peninentes  ont  ete  prises  en  consideration  et  les  documents 
modifies  en  consequence. 

Une  methode  de  tri  a  partir  des  criteres,  conditions  d’environnement  et  dominantes  a  ete  elaboree 
et  appliquee  a  l’ensemble  des  actes  elementaires  pour  ne  retenir  que  ceux  qui,  pour  la  robotisa- 
tion,  presentent  un  interct  operationnel,  qui  paraissent  robotisables  et  dont  la  frequence  d’occur- 
rencc  est  suffisante. 


ACTION  RETARDATRICE 
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Des  specifications  ont  ere  redigees  pour  chacun  des  acres  elementaires  rerenus  apres  ce  tri.  Des 
scenarios  tels  que  ceiui  illustre  par  la  figure  ci-dessus  onr  ere  elabores  de  fapon  a  situer  les  actes 
elementaires  rerenus  dans  un  environnemenr  operationnel  realiste  pour  satisfaire  les  besoins  de 
ingenieurs  charges  de  la  conception  des  robots. 


3/  ETAT  DE  L’ART  ET  MANUEL  DE  DIMENSIQNNEMENT. 


En  parallele  avec  l’etude  des  missions,  a  ete  etabli  un  “Etat  de  PArt”  rassemblant  les  elements  d’- 
infoimarions  regroupes  par  “Briques  technologiques”  d’interet: 

Locomotion,  mecanique,  energie. 

Capteurs 

Electronique 

Transmissions 

Architecture 

Precedes  et  technologies  d'armement 


Cet  etat  de  l’art  a  ete  conqu  sous  forme  de  base  de  donnees  infomiatique  contenant  des  fiches  si- 
gnaletiques  de  materiels  et  contenant  des  photographies  ou  des  croquis  tels  que  ceiui  du  robot 
“VERT,  vehicule  d’Exploration  Reconnaissance  Inspection  (de  zones  contaminees). 


Figui'e  4 


Cet  etat  de  1’an,  qui  rassetnble  des  objets  typiques,  constitue  un  rassemblement  d’exempies 
illustrant  lo  manuel  de  dimensionneraent  permettant  de  prevoir  les  dimensions  des  materiels  fu¬ 
ture,  en  s’appuyant  sur  des  statistiques  relatives  aux  materiels  existants,  traduites  en  formules  a 
chaque  fois  que  possible.  U  traite  essentiellemem: 

-  des  sources  d’energie  a  conversion  directe  et  indireete 

*  des  platefomies  mobiles,  roulames.  fkntantes  et  volames. 

*  des  sous  systemes  robotiques:  capreurs,  toureiles  de  vision,  manipulateurs. 

svsteme  de  controle  commande  portables,  ponatifs,  en  shelter. 

-  des  transmissions  hertriennes  ou  a  laisse  et  des  questions  de  discretion  associees. 
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-  des  precedes  specinquement  militaires  tels  que  la  vision  militaire  et  certains 
systemes  d’annes  juges  d’interet. 

4/  CONCEPTION  DES  ROBOTS 

Les  actes  elementaires  retenus  en  fin  de  premiere  phase  ont  ete  examines  en  utilisant  les  rensei- 
gnements  rassembles  au  coins  de  la  deuxieme:  des  projets  de  robots  specifiques  plus  ou  moins 
satisfaisants  ont  ete  proposes  pour  realiser  chacun  d’entre  eux.Ces  projets  couvrent  des  actes 
aussi  divers  que  la  transmission  d’alene,  le  chargement  et  dechargement  de  vehicules  er.  d’heli- 
copteres,  le  combar  rapproche,  diverses  operations  de  deminage  et  neutralisation  d’explosifs,  le 
balisage  d’un  itineraire,  diverses  operations  de  reconnaissance,  la  decontamination,  le  franchis- 
sement  des  cours  d’eau,  les  travaux  de  terrassement...  etc 

Souvent,  plusieurs  solutions  ont  ete  proposees  pour  un  meme  probleme  comme  le  montre  l’his- 
togramme  ci-dessous: 


Histogramme  donnant  le  nombre  de  solutions  differentes 
proposees  pour  les  68  actes  elementaires. 


Fig  5 


Une  analyse  tonctionnelle  et  statistiaue  effectuee  sur  les  paramerres  smvants: 

Deplacement 

Manipulation 

Captcurs 

Autonomie 

Retour  video 

Telecornmandc. 

a  pemtis  de  preciser  les  points  communs  des  differents  concepts  de  robots  specifiques.  en  vue 
de  la  conception  de  robots  plus  polyvalents.  Les  consequences  de  cette  analyse  ont  ete  doubles: 

-  On  s’est  rendu  cotnptc  que  le  rote  de  cenaines  techniques,  relics  que  par  exemplc  la 
telemanipulation,  etait  moins  imponant  que  prevu. 

-  A  1’inverse,  certaines  techniques  ont  vu  leur  poids  reiatif  augtnenter  considerable  - 
ment.  C’est  par  exernple  le  cas  de  la  teleoperation  par  objectifs*'  qui  apparait  dans  un  grand 
nombre  dc  cas,  comme  le  montre  la  siatistique  suivante  porunt  sur  68  cas  reperes  par  leurs  nu- 
raeros: 


1  *  System  de  teleoperaaon  pemeuaia  de  rut  transmettre  ou  mobile  control#  que  des  convnandes  rarefiees.  Ce  syste* 
m  devrait  aussi  s'aecommoder  d'm  retour  d'imaqes  ratenii.  Les  promts  de  robots  envisages  darts  Ntude  prennem  en 
sample  cette  iticessite. 
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Observation  reconnaissance  , 
Combat 

Amenagement  du  terrain 
Mines  et  explosifs 
Franchissement 
NBC 

Actions  actuellement  aeroportees 
Logistique 


23,  31,  32,  46,  47,  52,  53,  54 

05,  11,24,  25,  39 

20,  01,  38 

14,  15,  16,  42,  43 

29,  02,  37,  56,  57 

35,  51 

68,  58,  59,  60,  49,  66 
08,  09,  28,  50,  40 


Ainsi  a  ete  consume,  pour  completer  la  liste  des  robots  specifiques  a  chaque  acte  elementaire,  un 
catalogue  descriptif  de  “robots  polyvalents”  capables  de  plusieurs  actes  elementaires. 


Tous  les  actes  elementaires  n’ayant  cependant  pas  pu  etre  couvens  par  les  robots  polyvalents,  la 
liste  finale  des  robots  soumis  aux  operations  d’analyse  de  la  valeur  objet  de  la  phase  suivante 
comporte  a  la  fois  des  robots  specifiques  tels  qu’un  vehicule  de  transport  de  blesses  et  des  ro¬ 
bots  generiques:  vehicule  d’attaque,  vehicule  ouvTeur  de  breche, robot  lance  flammes,  robots  de- 
signateurs  d’objectifs...  ces  quelques  exemples  montrant  rapparition  de  “filieres  techniques”  en¬ 
trant  en  concurrence  en  vue  de  la  realisation  d’operations  relativement  analogues. 


A  ce  niveau  ont  ete  rediges  a  nouveau  des  scenarios  d’emploi  illustrant  les  nouveaux  concepts 
Ces  scenarios  ont  ete  choisis  de  raaniere  a  illustrer  et  valider  les  demandes  de  performances  de- 
mandees  par  les  responsables  de  l’etude  operadonnelle. 


Scenario  No  13 

Vehicule  ouvreur 
de  breche. 
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Figure  No  6 


Chaque  scenario  comporte  quatre  parties  principals: 

-  Situation, 

-  Mission 

-  Deroulement  comprenant  mise  en  place,  fonctionnement,  repartition. 

*  Conclusion  componant  un  premier  jugement  de  valeur.  indiquant  les  autres  em- 
plois  possibles  et  les  limites  d'eraploi.  evaiuant  les  raisons  d’echec  possibles. 


C6 . 6 


A  dtre  d’exemple,  la  conclusion  relative  au  robot  ouvreur  de  breche  indique: 

"  Le  franc kissement  des  positions  ennemies  sans  le  robot  aurait  pose  des  probldmes  relative- 
ment  longs  d  resoudre:  mise  en  place  d’appuifeu  d’artillerie  ou  de  mortiers,  attaque  des  posi¬ 
tions,  probablement  necessite  de  les  nettoyer. 

Le  robot  ouvreur  de  brie  he,  pour  echapper  aux  tires  de  l’  ennemi,  doit  etre  tris  mobile  et  puis- 
samment  blinde,  pouvoir  neutraliser  I’adversaire  par  des  sirs  massifs,  savoir  placer  des  fumi- 
g  fries  aux  endroits  necessaires. 

La  consommation  de  munitions  etant  forcemeat  tres  grande,  ilfaut  pouvoir  la  moduler  selon  les 
objectifs  a  trader:  la  ou  la  menace  est  plus  grande,  ilfaut  pouvoir  diriger  un  tir  plus  dense. 
L’emploi  de  ce  robot  necessite  une  nette  separation  entre  positions  amies  et  positions  ennemies 
pour  eviter  toute  meprise.” 


5/  PLANS  D’ACTION  A  MOYEN  TERME  ET  A  LONG  TERME 


La  conception  de  ces  plans  a  ete  prtictidtie  d’une  phase  d’analyse  de  la  valeur  comportant  un 
grand  nombre  d’ operations  de  notations  ou  de  chiffrage  se  rapponant  a  chacun  des  concepts  de 
robots  retenus  pour  le  criblage  final. 

Au  cours  de  cette  analyse,  la  plupart  des  concepts  ont  ete  legerement  optimises  de  maniere  a 
pouvoir  realiser  les  missions  prtivues  au  moindre  cout,  chaque  responsable  de  concept  avant 
pour  objectif  d'obtenir  la  meilleure  notation  possible  en  fonction  des  criteres  utilises  pour  classer 
ces  robots.  Ces  criteres,  eux  memes  divises  en  sous-criteres,  sont  essentiellement  les  suivants: 

-  Criteres  operationnels:  performances,  nouvelles  possibility  operationnelles, 
contraintes,  taux  d’utilisation,  securite,  vulnerability. 

-  Criteres  humains:  Danger,  ptinibilite,  gain  en  personnel,  gain  en  personnel  qualifiti, 

stress. 

-  Critdres  techniques:  complexity,  delai  de  recherche,  developpement,  indus¬ 
trialisation. 

-  Criteres  financiers:  eouts  recurrents,  couts  de  possession  lies  a  la  complexity. 

Chacun  des  robots  a  ete  soumis  a  une  serie  de  notations  ou  de  chiffrages  pour  pouvoir  qualifier 
son  niveau. 

L'interet  general  operationnel  ou  humain  est  represente  par  la  moyenne  des  notes  obtenues  pour 
chacun  ties  criteres.Les  coefficients  attribues  a  ces  notes  avaient  ete  prealablement  soumis  a  un 
panel  d’une  dizaine  d’officiers. 


On  a  reussi  a  atmbuer  a  chaque  robot  les  memes  questions,  mais  il  s’est  trouve  necessaire  d’at- 
tnbuer  des  grilles  de  notations  differentes  a  chacune  d’entxc  elles:  les  helicopteres  teleoperes  de 
reconnaissance  et  les  cmrins  do  franchissement  du  Genie  ont  dus  etre  notes  en  utilisant  des  ba- 


remes  differents. 
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Le  premier  cvqui  presente  un  curactere  eliminatoire,  a  ete  effectue  en  utilisant  la  grille  de  la  fi¬ 
gure?:  ..  . 


CRITERES  OPERATIONNELS 
Figure  7 


Des  symboles  appreciates  ont  ete  preferes  aux  notes  pour  rencire  le  tableau  plus  parlant: 

F=  faible 
M=  moyen 

B-  =  reiativement  bien 
B+  =  franchement  bien 
TB  =  tres  bien. 

La  figure  7  indique  les  noras  des  principaux  robots  selectionnes.  II  s’agit  essenriellement  d’en- 
gins  de  combat,  de  reconnaissance  et  observation,  de  logistique. 

Deux  rnodeles  se  distinguent  particulierement,  il  s'agit  d’un  helicoptere  radiocommande  de  re¬ 
connaissance  (HelicoRad)  et  d’un  robot  antichar  tRobAC). 
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La  raisabilite  economique  a  ete  veririee  au  moyens  des  parametres  apparaissant  figure  8. 


Les  couts  figurant  sur  la  grande  grille  sont  des  couts  marginaux  qui  ne  prennent  pas  en  compte 
l’amortissement  des  recherches  et  des  etudes.  Les  couts  de  maintenance  n'ont  pas  ete  evalues  di- 
rectement:  on  a  admis  qu’ils  etaient  proportionnels  a  un  facteur  de  complexite  ayant  egalement 
ete  utilise  lors  de  l’analyse  operationnelle. 

Les  couts  et  delais  de  recherche,  deveioppement  et  industrialisation  Pont  ete  a  partir  de  plannings 
types  reproduisant  {’experience  acquise  lors  de  la  realisation  de  robots  mobiles  civils.  avec  cer¬ 
tains  amenagenaents  pour  prendre  en  compte  les  exigences  particuiieres  aux  fabrications  d’ane  • 
ment. 

Ces  diagrammes  pennettent  d’orienter  les  premiers  choix,  a  condition  de  se  souvenir  que  le  cout 
d’un  programme  est  normalement  plus  faible  que  la  somme  des  couts  de  ses  constituants,  les  re¬ 
cherches  et  les  investissements  etant  susceptibles  de  s’epauler  rautuellement. 

Parmi  les  concepts  proposes,  certains  sont  realisabies  a  relativement  court  terme:  il  s’agit  essen- 
tiellement  de  machines  existant  deja  sur  le  marche  tels  que  par  exemple  les  robots  de  desamorpa- 
ge  d’explosifs  ou  des  robots  de  nettoyage  susceptibles  d’etre  utilises  pour  faire  de  la  decontami¬ 
nation. 
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Mais  1’objectif  de  l’etude  couvre  avant  tout  le  moven  et  le  long  terme.  La  figure  9  presente  la  me- 
thodologie  d’etablissement  de  ces  plans. 

La  recherche  a  moyen  terme  a  pour  but  essentiel  1’ acquisition  des  technologies  necessaires  pour 
atteindre  les  objectifs  retenus.  Lorsque  ces  objectifs  de  connaissance  seront  atteints,  il  faudra  en¬ 
gager  des  developpements  industries  adaptes  aux  produits. 


La  figure  10  foumit  des  informations  plus  detaillees  sur  la  methode  de  determination  des  pro¬ 
grammes  de  recherche.  H  a  ete  considere  que  la  validite  des  recherches  devait  etxe  verifiee  grace 
a  des  demon strateurs  physiques  venant  en  complement  des  recherches  papier.  Ces  demonstra- 
teurs  ne  doivent  pas  etre  confondus  avec  des  maquettes  souvent  utilisees  par  les  chercheurs  pour 
valider  leurs  methodes  de  prevision.  Ils  seront  beaucoup  plus  orientes  utilisation  et  ressemble- 
ront  deja  a  des  robots  de  combat,  mais  ils  ne  representeront  pas  encore  des  robots  optimises  pret 
al’emploi. 
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IDENTIFICATION  DES  TECHNOLOGIES  DE  BASE 
A  DEVELOPPER 


/IDENTIFICATION  ET  > 
REGROUPEMENT  DES 
iDEMONSTRA  T E  URS 


Figure  10 


La  figure  10  montre  comment  quelques  demonstrateurs  permettent  de  couvrir  la  gamme  des 
technologies  sans  pour  autant  construire  la  totalite  des  systemes  envisages.  Les  fonctions  et  sous 
systemes  sont  listes  pour  chaque  robot  suivant  les  colonnes  d’une  matrice  comportant  une  ligne 
par  robot  retenu.  Ainsi,  certains  systemes  appartenant  h  deux  types  de  robots,  ne  seront  pas 
etudies  deux  fois:  le  systeme  de  teieconduite  d’un  robot  de  combat  peut  par  exemple  convenir  au 
char  lourd  de  deminage  dont  les  performances  en  vitesse  sont  moins  elevees.  Les  systemes 
d’autodefense  de  vehicule  sont  adaptables  aux  robots  fixes,  les  systemes  de  transmissions,  cer¬ 
tains  capteurs  particulierement  difriciles  tels  que  les  telemetres  imageurs  etc...  ne  seront  etudies 
qu’une  seule  fois,  mais  seront  essayes  sur  quelques  engins  differents  dans  des  conditions  diffe- 
rentes. 

Parmi  les  recommandadons  qui  figureront  dans  le  rapport  final,  se  trouveront  tres  probabiement 
les  suivantes: 

*  Circonscrire  les  etudes  aux  robots  vraiment  les  plus  interessants,  les  recherches  prevues  etant 
souvent  lourdes  et  susceptibles  de  reveler  des  difficultes  imprevues  diminuant  l'interet  des 
concepts  concemes:  une  marge  dinteret  est  necessaire  pour  eviter  les  impasses. 

Cette  recommandation  n'exclut  pas,  au  contraire,  une  verificauon  periodique  du  bien  fonde  des 
objecufs. 
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Elle  n'exclut  pas  non  plus  la  realisation  immediate  par  l'industrie  des  robots  utiles  et  faisables  a 
coun  terme,  tels  que  les  "Minirobots''. 

*  Introduce  une  certaine  modularity  dans  les  concepts  pour  limiter  les  etudes  specifiques,  les 
moyens  industries  necessaries  et  la  quandte  de  materieis  a  emporter  par  les  utilisateurs. 

*  Attacher  une  grande  imponance  aux  etudes  de  transmissions  d'une  part,  aux  etudes  d'autono- 
mie  d'autre  part,  pour  faciliter  l'emploi  de  nombreux  robots  dans  un  meme  compartiment  de  ter¬ 
rain. 

*  Les  developpements  concemant  1’  “armement”  sont  de  loin  les  plus  lourds  en  prix,  mais  ils 
ne  conditionnent  le  delai  que  dans  deux  cas  importants  concemant  un  robot  d’autodefense  et  un 
robot  de  tri.  Par  contraste,  les  recherches  concern?^;  la  teleconduite  des  vehicules  sur  objectifs 
menent  en  oeuvre  des  budgets  relativement  faibles  mais  ne  peuvent  pas  etre  realisees  en  un 
temps  tres  court,  le  franchissement  d’une  etape  necessitant  le  succes  soigneusement  verifie  et 
consolide  de  l’etape  precedente.il  est  done  recommande  de  commencer  immediatement  ces  re¬ 
cherches  peu  cheres  mais  essentielles  pour  la  grosse  majorite  des  projets. 

*  Une  competition  enre  filieres  techniques  existe  entre  les  moyens  de  reconnaissance:  vehicule 
chenille  classique,  vehicule  rapide  a  creer  et  helicoptere  telecommande,  le  vehicule  classique  se 
presentant  a  la  fois  comme  le  plus  couteux  et  le  plus  difficile  a  realiser.  II  n”est  pas  propose  de 
resoudre  ce  dilemme  avant  les  cinq  annees  a  venri,  les  etudes  de  base  etant  les  memes  et  exigeant 
au  moins  ce  laps  de  temps. 

*  Un  demonstrateur  de  teleoperation  sur  objectifs  de  vehicule  tout  terrains  sera  indispensable 
pour  prolonger  les  premieres  etudes  VERI  jusqu’a  un  niveau  d’ applicability  au  problemes  mili¬ 
taries. 

*  Differents  robots  antichar  correspondant  aux  memes  besoins  operationnels  mais  realises  de 
fagon  tres  differente  pourraient  etre  deveioppes.  Trois  axes  de  recherche  apparaissent  neces¬ 
saries: 

-  Tri  depone  compatible  avec  transmission  ralenrie  des  images. 

-  Chaine  d’ identification  et  de  riposte 

-systeme  d’ identification  des  amis  utilisabie  par  les  engins  terrestres. 

*  La  teleoperation  des  armes  utilisera  des  precedes  specifiques,  ne  reprenant  qu’en  p  “tie  les 
technologies  de  teleoperation,  de  plus  ces  procedes  devrontetre  couples  de  faqon  active  avec  les 
procedes  de  conduite  des  vehicules.  Un  couplage  "passif ’.  par  rintermediarie  de  capteurs  enre- 
gistrant  simplement  les  mouvements  du  vehicule  ne  suffira  pas  dans  tous  les  cas  pour  obtenir  les 
vitesses  d’action  et  les  precisions  souhaitees.  La  prise  en  compte  des  ordres  du  systeme  de  pilo¬ 
tage  par  le  systeme  d’anmes  permettra  seule  d’obtenti  les  performances  souhaitees. 

*  L’analyse  de  la  valeur  operationnelle  a  montre  que  la  taille  des  systemes  de  comniande  etait  un 
facteur  imponant  pour  les  operationnels.  Un  gros  travail  de  developpement  de  ces  interfaces  de 
masse  reduite  est  necessaire.  Les  techniques  dites  de  “telepresence",  non  appliquees  aujourd’hui 
aux  vehicules  terrestres  devront  farie  l’objet  d’une  attention  particuliere. 

61  CONCLUSION 

Cette  etude  de  concepts  de  robots  de  combat  nous  a  permis  de  cemer  les  besoins  de  l’Armee  de 
Terre  tels  que  connus  aujourd’hui  et  de  jeter  les  bases  d’un  programme  de  recherche  tendant  a 
rendre  realisables  des  concepts  relativement  futuristes. 

Les  programmes  proposes  nous  paraissent  valides  pour  une  periode  d’environ  4  a  6  ans  sui- 
vant  les  points  concemes.Les  causes  de  modifications  previsibles  sont  les  suivantes: 

Modification  des  besoins  pergus  par  les  operationnels 

Apparition  de  nouvelles  technologies  modifiant  les  faisabilites  et  les  couts. 

Nous  proposons  done  une  remise  a  jour  de  la  presente  etude  dans  trois  ans. 
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ROBOT  TRIALS: 

DARDS  TEST-BED 

an  APPROACH  for  ROBOTS  TESTING 

%  J.Ph  QJJIN 

DASSAULT  ELECTRONIQUE 
55,  guai  Marcel  DASSAULT 
92214  Saint-Cloud 


ABSTRACT 

The  use  of  mobile  surveillance  and  intervention  robots  is  necessary  in  a  hostile 
environment  where  human  beings  cannot  complete  a  mission  without  danger.  In  few  years, 
one  may  expect  that  a  robot  will  be  intelligent,  as  far  as  it,  or  he,  will  be  able  to  react 
with  and  against  its  environment. 

However,  and  for  many  reasons,  a  robot  must  be  robust.  Reliability  of  equipments, 
survivability  in  an  agressive  and  hostile  environment  are  the  keywords.  But  how  test,  and 
certify,  this  robustness?  Advanced  technologies  in  robot's  world  could  imply  use  of 
symbolic  computing,  neural  nets,  genetics  algorithms,  just  for  today...  The  probably  non 
short  tern  deterministic  behavior  of  the  robot  may  not  be  convenient  for  massive  testing  at 
integration  and  trial  phases,  as  far  as  the  robot  do  have  to  react  against  the  outside  world. 

So  far,  this  document  does  not  describe  existing  solutions  for  such  a  challenge,  but  do 
expose  what  real  problems  could  be  encountered  during  the  testing  phases. 
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PARTIS  TEST-BED 

DARDS,  an  acronym  of  Dimonstrateur  Autonome  a  Rapidite  de  Deplacement  pour  la 
Surveillance,  is  a  French's  DoD  major  program.  It  is  mainly  funded  by  DRET  and  is 
designed  to  be  a  test-bed  for  advanced  complex  functions  and  sensors. 

It  is  not  designed  to  be  a  fully  operational  system.  However  the  robot  include  methods  and 
technologies  already  existing  in  others  "off  the  shelf  systems  and  provide  many  room  for 
future  experiences:  power  supply,  standard  mechanical  interface,  wide-band  digital  and 
video  links...  Fig  1  &  fig  2  present  DARDS. 

During  the  trials,  DARDS  is  fully  supported  by  a  separate  mobile  control  center.  It  is 
integrated  on  a  stand-alone  25  ft  truck,  equiped  with  four  control  stations.  These 
workstations  are  designed  for  real  time  control  and  easy-to-use  system.  The  control  center 
is  backuped  by  an  autonomous  power  supply. 


ELEMENTARY  TESTS 

A  common  method  of  qualifying  a  system  consists  in  Performing  elementary  tests  which 
become  increasingly  complex.  Mobility,  Agility,  Intervention  effectless...  This  classic 
methodology  is  already  used  for  robot  integration  (fig  3). 

These  initial  tests  are  essential  for  checking,  both  simply  and  without  contusion  between 
parameters,  the  quality  of  the  various  algorithms  employed.  Each  trial  must  be  designed  to 
answer  a  question  concerning  the  correct  execution  of  a  specific  function. 

The  tests  of  the  basic  functions  must  therefore  be  combined  in  a  dynamic  and  coherent 
manner  to  make  an  in-the-lab  evaluation  of  the  future  product.  The  definition  of  basic 
action  in  the  integration  process  and  trials,  in  a  more  open  world,  is  a  fundamental 
question. 


DEFINITION  of  TRIALS 

With  an  environment  which,  by  definition  for  a  mobile  autonomous  robot,  ?s  poorly 
known,  the  exploration  of  possible  cases  is  difficult  and  of  course  costly.  It  can  also  be 
inachievable. 

Several  methods  may  be  considered  as  a  random  approach,  where  the  operator  imagines 
complex  cases  of  intervention.  A  combination  approach,  all  possible  cases  are  described 
with  all  possible  problems  to  write  it  and  keep  it.. .A  statistical  approach  for  which  cases 
apparently  more  significant  are  extracted  from  the  previous  analysis  (fig  4). 

These  methods  are  effective  in  the  case  of  elementary  tests,  but  are  difficult  to  apply  in  the 
case  of  weighted  combinations  of  these  tests.  By  the  end,  there  is  a  danger  of  subjective 
bias  linked  to  the  operator. 
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FIGURE  1 


C7.5 


AC/243-TP/3 


FIGURE  3 


AC/243-TP/3 


C7.6 


TRIALS 

DEFINITION 


RANDOM 

APPROACH 


0 


NON-EXHAUSTIVE 


0 

PROBABLY 


risk  01  The  BEST  solution 

but  TOO  LONG  & 
OPERATOR'S  BIAS  too  EXPENSIVE 


METHODS  PARTIALLY 
APPLICABLE  to  ROBOT’S 
TRIALS 


FIGURE  4 


STATISTICAL 
!  APPROACH 


DIFFICULT  to  USE 
as  far  as 

MANY  PARAMETERS 
MUST  CHANGE 
at  the  SAME  TIME 


AC/243-TP/3 


*■  ? ' f*£ : ’vtrfB ? ■  >.  ■  _££; fsrv; 


C7.7 


AI  and  future  techniques 

AI  techniques  are  more  and  more  introduced  to  achieve  appropriate  "intelligence"  in  a 
robot.  The  actual  human  intelligence  can  be  approached  by  a  three-part  synthesis  of 
symbolic  process,  connectionist  process  and  genetic  algorithms  (fig  5). 

In  the  olden  time,  the  classical  feature  of  a  system  was  based  on  deterministic  algorithms. 
AI,  or  symbolic  process,  knowledge  based  system,  can  replicate  human  expertise  within 
narrow  domains.  But  this  approach  is  a  human-related  process,  subject  to  a  human  bias 
and  human  ability  to  foresee  the  environment  that  the  robot  will  encounter.  Moreover, 
expert  system  are  often  non-deterministic  in  time. 

Conneetionist  process,  implemented  in  neural  networks,  could  be  suited  for  bug's 
intelligence.  Actual  works  on  such  technology  is  in  progress.  The  results  may  be  non 
deterministic  if  an  imperfect,  or  incomplete,  learning  is  made. 

Genetic  algorithm  is  a  very  promising  technique.  It  is  based  on  self-modifying  algorithm 
with  a  feed-back  mechanism. 

All  these  techniques  seem  to  be  the  future  of  intelligent  robot,  allowing  it  to  learn  from 
and  adapt  to  uncertain  and  changing  environments.  But  integration  and  trials  are  quite 
more  complicated  as  far  as  it  costs  a  lot  to  test  a  robot  on  a  proving  ground,  and  it  costs 
much  more  to  test  a  system  with  an  unpredictable  answer  to  its  environment  (fig  6). 


SIMULATION  of  TRIALS 

A  trial  specification  aid  tool  may  partly  solve  the  problem  raised.  This  approach  consists 
in  modelling  robot  tunctions  and  modelling  the  environment  together  with  protagonists 
and  factors  present  therein  (fig  7). 

Complete  modelization  includes  simulation  of  robot  mobility,  of  reception  sensors  and 
associated  processing  and  finally,  simulation  of  the  terrain  and  elements  on  the  terrain, 
wether  fixed,  pseudo-mobile  and  mobile.  The  knowned  behavior  of  these  objects,  or 
actors,  must  be  included  in  the  simulation. 


SIMULATION  of  the  ROBOT 

The  mobility  function  may  be  simulated  by  a  dynamic  vehicle  model  taking  into  account 
the  mechanical  characteristics  of  the  vehicle.  The  navigation  and  localization  function  may 
also  be  simulated.  But  die  most  complex  is  lire  simulation  of  the  perception. 

Fig.  8  show  how  could  be  built  tire  simulation  for  perception.  It  is  of  course  die  most 
complex  simulation,  and  it  can  be  achieved  in  three  different  ways: 

*  by  direct  use  of  operational  algorithms  on  a  real  image.  This  solution,  theorically 
the  most  accurate,  has  the  main  disadvantage  of  generation  or  an  image  as  close  as 
possible  to  the  robot's  environment. 

*  by  considering  that  most  image  processing,  at  a  low  pre-processing  level,  is  based 
on  simple  algorithm  (SQEBEL  filter  by  example).  A  synthetic  Unage  may  be  built 
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identical  to  a  filtered  image  thereby  allowing  the  application  of  processing  and 
interpretation  algorithm. 

*  by  indicating  to  the  robot  how  it  could  or  should  perceive  and  interpret  the  various 
objects  in  the  scene.  Of  course,  this  last  solution  is  less  costly  in  calculation  time,  but  can 
generate  an  artefact. 

The  robot  could  learn  tire  way  the  operator  detect  the  objects  by  using  a  neural  network. 
Fig  9  and  fig  10  shows  the  organization  of  terrain  simulation. 


CONCLUSION 

Simulation  of  the  trials  to  be  performed  in  order  to  qualify’  mobile  robots  does  r.ot 
eliminate  the  basic  testing  of  each  function.  New  techniques,  connectionist  and  genetic 
algorithms,  make  the  system  far  more  difficult  to  test  and  can  generate  non-deterministic 
features.  However,  the  robustness  of  the  robot  must  be  qualified  and  quantified.  This  a 
new  challenge  in  integration  and  test  process. 


]  ROBUSTNESS  for  AUTONOMOUS  VEHICLES,  Robert  FINKELSTEIN,  AUVS, 
Spring '90. 
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Klaus-Peter  Holzhausen,  Dr.-Ing. 
and  Hans-Ludwig  Wolf,  Dipl.-Ing. 
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Neuenahrerstr .  20 
5307 Wachtberg-Werthhoven . 

Germany 
(228) 852-480 


1.  INTRODUCTION 

Military  Telerobotic  Systems  for  complex  missions  will  be 
Kuman-Machine-Systems  at  least  for  the  next  decades.  These 
"robots"  will  be  semiautonomous  and  have  to  be  coupled  to  the 
human  operator  (NATO,  DRG,  Panel  1  Study  on  "Robotics  in  the 
Battlefield",  1985,  Study  on  "Robotic  for  the  Federal  Armed 
Forces",  Research  Establishment  for  Applied  Sciences  FGAN,  3onn, 
Germany,  1986,  and  Dornier  Study  on  a  "Military  Robotics 
Programme",  1987)  . 

Increasing  overall  system  performance  of  military  Human- 
Machine-Systems  is  only  possible  by  adapting  technical 
components  to  humans  (Human  Engineering)  .  This  basic  principle 
must  be  part  of  the  early  phase  of  any  weapon  development  .Human 
Performance  is  practically  defined.  Telepresence  in  military 
robotics  systems  consisting  of  control  station,  data  link,  and 
the  robot  vehicle  is  affected  by  the  ease  of  interaction  in  a 
"virtual  workspace"  .  Information  requested  by  the  control 
station  operator,  his  cognitive  support  by  expert  systems  as 
well  as  the  design  of  the  interaction  between  distributed 
decision  levels  of  robot/control  stations  are  human  engineering 
topics  that  are  decisive  to  system  performance.  It  becomes 
increasingly  important  when  the  data  link  bandwidth  between 
robot  and  control  station  is  reduced  for  other  military  reasons. 

The  Research  Institute  for  Human  Engineering  (FAT) ,  Bonn, 
Germany,  is  a  research  institute  of  the  German  MoD.  It  is 
specialised  in  evaluation  and  design  of  military  Human-Machine- 
Systems  and  responsible  for  research  of  the  methodology  of  human 
engineering  evaluation  of  military  systems.  FAT  has  20  years 
experience  in  the  simulation  of  Human-Machine-Systems. 

Laboratory  experiments  in  simulators  'using  soldiers  as  subjects 
are  performed  under  controlled  experimental  conditions.  The 
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conditions  can  be  exactly  reproduced  and  render  results  that  are 
suitable  for  statistical  treatment.  Those  laboratory  experiments 
are  most  suitable  for  parametric  studies  at  low  cost .  The 
■experiments  often  lead  to  the  detection  of  critical  system 
'■'elements  which  limit  system  performance.  Another  human 
engineering  issue  is  a  suitable  function  allocation  between  man 
and  machine  as  well  as  the  development  of  quantitative  methods 
for  evaluation  of  future  system  concepts . 


MTLTTARY  ROBOTIC  RESEARCH  PRQCRAMMES  IN  GERMANY 


The  FAT  research  in  Human-Machine-Systems  includes 
activities  in  the  field  of  telerobotics .All  research  activities 
are  focussing  on  land  robots.  These  robots  are  more  complex  in 
comparison  to  air/sea  systems  because  they  operate  in  a  complex 
environment.  Obstacles  and  topographies  require  friend/ foe 
detection  and  tactical  decisions  in  very  limited  time.  The  FAT 
research  activities  are  divided  into  four  phases  (Figure  1) .  The 
first  phase  is  the  EROS  phase.  EROS  is  a  laboratory  test  site 
for  an  Experimental  Robot  System.  EROS  is  used  to  assess  system 
components  that  limit  performance  in  a  robot  system  consisting 
of  a  robot  vehicle  in  a  test  environment  and  an  operator  in  a 
control  station.  It  is  designed  to  measure  system  response, 
compare  control  concepts,  search  for  performance  bottle  necks, 
define  ergonomic  constraints,  and  serve  as  a  test  bed  for  a  wide 
variety  of  robotic  subsystems . 

System  simulation  is  a  parallel  and  simultaneous  approach  to 
the  development  of  the  hardware  system.  Simulation  models  of 
both  the  robot  and  the  experimental  environment  are  developed  as' 
part  of  the  SIMROS  programme  for  parameter  studies  with  a  setup 
equivalent  to  the  hardware.  SIMROS  also  uses  different  system 
components  than  those  at  hand  in  hardware.  System  performance  of 
different  technical  solutions  can  thus  be  studied,  including  an 
experimental  situation  consisting  of  a' human  operator  at  a 
control  station  managing  both  the  hardware  robot  and  a  simulated 
robot  to  tackle  problems  of  distributed  decision-making  in 
distributed  systems. 

Two  more  experimental  project  phases  are  planned.  The  NAVROS 
approach  includes  expert  system  approaches  for  shared  decision 
making  in  a  cooperative  environment.  Theoretical  and 
experimental  studies  are  planned' for  a  system  capable  of 
semiautonornous  and  autonomous  phases  of  operation  under 
hierarchical  operator  management.  The  TAKROS  phase  includes 
tactical  options  influencing  the  robot  operation  in  a  hostile 
environment.  'v 
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The  EROS  project  focusses  on  ergonomic  aspects  of  the  Human- 
Machine-Interface  while  guiding  a  telerobotic  vehicle  in  a  test 
environment.  It  is  based  on  a  vehicle  that  operates  in  a  factory 
hall.  The  factory  hall  measures  approximately  18  x  14  m.  It  is 
equipped  with  cvlindric  obstacles  of  about  0.50  m  diameter  and 
1.50  m  height.  Also  included  are  facilities  for  data 
transmission  to  and  from  the  experimental  vehicle  and  a  TV 
Gigahertz  link.  The  hall  is  equipped  with  a  number  of  safety 
systems  to  insure  safe  operation  of  the  vehicle.  The  vehicle 
itself  (Figure  2)  is  a  four  wheel  platform  of  less  than  2  m  in 
length.  It  is  1.30  m  wide  and  about  1.60  ra  high.  The  vehicle 
has  two  electric  motors  that  power  the  two  back  wheels.  It  has  a 
pair  of  front  wheels  which  are  steerable.  The  maximum  speed  is 
about  2 . 3m/s . 


I 

I 


Figure  2:  EROS  test  vehicle 

The  EROS-vehicle  is  equipped  with  five  cameras.  One  of  the 
cameras  is  mounted  in  the  rear  of  the  car  on  a  pan  and  tilt 
head.  This  pan  and  tilt  head  looks  mainly  straight  ahead  on  the 
driving  path.  Two  cameras  are  mounted  in  the  body.  Through 
mirrors  they  support  views  of  the  left  and  right  vehicle 
contours.  These  cameras  are  useful  for  manoeuvering  tasks  such 
as  operating  through  a  narrow  lane.  Two  more  cameras  look 
backwards  respectively  directly  in  front  of  the  car.  Ail  of  the 
cameras  can  be  switched  to  the  TV  link  as  raw  videos  or  in  a . 
mixed  version  as  described  later. 
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Sight  ultrasonic  sensors  (Figure  3)  are  mounted.  Three  of  them 
look  forward,  two  look  left  and  two  right.  The  other  sensor  is 
mounted  looking  backwards . The  sensors  have  a  measuring  range  of 
0.3  to  10.7  m.  The  resolution  is  1%  over  the  whole  range.  Each 
of  the  sensors  has  an  angular  range  of  40° .  The  ultrasonic 
devices  are  safety  devices  that  automatically  reduce  vehicle 
speed  or  stop  it  if  the  distance  between  obstacles  and  the 
vehicle  decrease  beyond  defined  limits.  An  on  board  computer  is 
responsible  for  data  exchange  with  the  control  station,  visual 
system  and  on  board  equipment . 


Figure  3  Ultrasonic  system  on  the  EROS  vehicle 

The  control  console  has  a  main  TV  monitor  to  display  camera 
information  transmitted  from  the  vehicle.  This  TV  picture  has  a 
number  of  mixing  features  and  overlay  graphics  that  are 
described  later.  Additional  monitors  display  process  control 
information  in  switch  selectable  full  graphics  format. 

Inputs  to  the  vehicle  are  through  a  variety  of  control 
devices  which  include  controlsticks  and  an  optoelectronic  sensor 
ball.  Also  supplied  are  virtual  keyboards  and  function 
keyboards.  The  experimenter  uses  TV  cameras  which  are  mounted 
under  the  ceiling  and  on  the  walls  of  the  EROS  hall  for  safety 
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purposes.  He  also  uses  a  multiwindow  workstation  to  control  the 
operation  and  the  experiments. 


4.  VEHICLE  GUIDANCE 

The  EROS  vehicle  is  guided  through  a  wireless  link  from  the 
control  station.  Primary  control  is  through  one  of  the  before 
mentioned  control  devices  with  visual  feedback  through  the  TV 
channel  using  the  TV  monitor  (Figure  4) .  The  TV  picture  can  be 
presented  either  in  color  or  in  black  and  white.  Standard  vie¬ 
wing  uses  the  main  camera  on  the  pan  and  tilt  head.  Height  from 
the  ground  is  1.40  m  with  the  camera  looking  forward.  Camera 
deflection  is  25°  from  the  horizontal.  Switch  selectable  are 
views  through  the  backward  looking  camera  and  a  camera  which  is 
positioned  in  the  front  of  the  chassis. 


Figure  4:  EROS  main  display 

Additionally,  two  views  along  the  left  and  right  vehicle 
contours  can  be  mixed  into  the  lower  third  part  of  the  TV 
picture.  As  field  of  view  limitations  showed  degradation  in 
driving  performance,  a  wide  angle  lens  of  6.5  mm  with  an  angle 
of  ~  90°  was  selected.  Driving  while  panning  the  camera  is  very 
difficult  because  drivers  easily  lose  orientation.  This  state  of 
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disorientation  can  be  alleviated  by  azimuthal  camera  movements 
linked  with  the  deflections  of  the  steered  pair  of  wheels.  Thus 
a  "look  into  the  curve"  was  realized  which  showed  better  results 
in  precisely  following  a  curved  path. 

Another  increase  of  performance  was  achieved  by  overlaying 
graphical  information  in  the  video  display  (Figure  4)  .  This 
graphical  information  is  iconic  respectively  contains  simple 
graphic  features  that  do  not  overload  the  visual  scene.  In  the 
upper  third  of  the  display  three  overlays  were  positioned.  The 
left  overlay  symbolizes  a  steering  wheel  much  like  one  in  a  car. 
This  steering  wheel  represents  the  steering  inputs  as  commanded 
by  the  controlstick  x-axis  deflection,  transmitted  and  received 
as  true  steering  data  from  the  vehicle.  The  symbol  of  the 
steering  indicator  is  color  coded  to  serving  the  operator  as  an 
indication  of  safe  driving  and  steering. 

The  symbol  in  the  upper  right  area  represents  a  speedometer. 
Its  scaling  is  in  cm/sec.  The  needle  is  displayed  in  green  color 
curing  normal  operation.  Its  color  changes  to  yellow  if  the 
vehicle  control  processor  calculates  an  unsafe  speed  regarding 
the  curve  diameter  being  operated  upon.  During  backing  up  the 
vehicle  and  still  displaying  the  front  view  the  needle  is 
switched  to  red  thus  indicating  a  mismatch  between  display  and 
operation. 

In  the  middle  position  is  a  symbolic  top  view  of  the 
vehicle.  The  four  triangles  at  the  vehicle  corners  can  warn  that 
tha'.  corner  is  too  near  an  obstacle  or  a  wall  of  the  EROS  test 
facility  hall.  The  triangles  are  not  displayed  under  normal 
concitions.  They  are  displayed  in  yellow  color  if  a  distance  of 
2  m  is  reached.  Below  1  m  they  change  to  red  color.  The  distance 
measurement  is  achieved  by  the  ultrasonic  devices  (Figure  3)  .  If 
the  operator  does  not  command  a  change  of  directions  or  a  full 
stop  the  vehicle  is  automatically  stopped  by  a  hardware  security 
system  at  a  preprogrammable  distance  from  the  closest  obstacle. 

The  three  rectangles  at  the  lower  left  and  right,  sides  of 
the  vehicle  symbol  and  in  the  upper  center  (symbol  for  the 
wheels)  represent  the  state  of  drives  and  steering  system.  Green 
color  indicates  normal  operation  of  the  on  board  systems,  red  is 
a  warning  due  to  malfunction.  A  malfunction  in  these  primary 
driving  subsystems  causes  the  vehicle  to  stop.  For  further 
information  the  operator  uses  the  Process  Monitoring  graphics 
System  (PROMO)  on  the  side  monitor. 
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The  arrow  in  the  vehicle  symbol  symbolizes  the  rotation  of 
the  main  TV  camera  to  the  left  or  right  with  respect  to  the 
longitudinal  axis  of  the  vehicle. 


The  overlay  in  the  central  part  of  the  display  is  a  channel 
display.  It  is  calculated  and  projected  in  the  visual  scene 
showing  the  path  being  travelled  for  the  next  four  seconds 
assuming  constant  speed  and  steering  deflection.  For  steady 
cruise  the  channel  display  is  static.  In  the  case  of  steering 
into  a  curved  path  or  accelerating  the  channel  display  follows 
in  real  time.  It  is  a  help  during  normal  operating  conditions 
but  tends  to  overload  the  picture  during  manoeuvers  in  order  to 
maintain  a  curved  path  at  high  speed  accurately.  Further 


experiments  will  be  conducted  w. 


h  a  display  algorithm  using  a 


low  path  filter. 


Overlay  information  of  a  map  display  was  less  promising 
because  of  the  completely  different  viewpoints.  "Inside  out"  and 
"outside  in"  views  cannot  be  mixed  due  to  their  incompatibility. 
A  plan  view  is  thus  presented  on  the  side  monitor  upon  request. 
It  is  part  of  the  PROMO  System  that  helps  the  operator  to 
display  any  system  parameters  required  in  graphical  forms. 


This  PROMO  System  which  uses  two  side  monitors  is  a  very- 
efficient  tool  for  operator-robot  interaction.  Besides  map 
information  a  complete  set  of  both  graphics  and  alphanumeric 
status  information  can  be  called  up.  Moreover  results  cf  an 
expert  system  based  path  planner  are  displayed.  The  system 
stores  experimental  data,  retrieves,  and  displays  processed 
data.  Mode  selections  and  presettings  are  entered  via  a  menu 
oriented  virtual  keyboard  or.  the  screen.  Figure  5  shows  a  PROMO 
display  of  ultrasonic  data  as  measured  by  the  vehicle  sonar 
subsystem,  PROMO  is  a  muitiwindow  display  system.  For  this 
reason  the  ultrasonic  display  can  be  presented  together  with 
speed  and  status  information,  a  digitally  generated  ground  map, 
and  other  selected  system  data. 


A  variety  of  controls  is  available  for  vehicle  operation. 
All  controls  are  used  for  driving  and  to  input  information 
through  the  Human-Computer-Interface.  A  steering  wheel  is 
unsuitable  for  computer  inputs  and  was  therefore  disbanded. 
Standard  control  device  is  a  two  degree  of  freedom  finger 
operated  controlstick.  This  controlstick  is  built  into  a  small 
console  which  can  be  positioned  by  the  operator  at  his  work 
place  and  which  also  contains  function  switches  for  vehicle  and 
display  control.  The  controlstick  is  a  spring  centered 
displacement  controlstick.  The  x-axis  deflection  is  used  to 
command  right  or  left  steering  wheel  deflections,  the  y-axis 
deflections  are  for  speed 
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Figure  5:  PROMO  sensor  display  window 

control  in  forward  or  reverse  directions.  Another  controistick 
w^th  similar  characteristics  is  available  to  control  the  camera 
pan  and  tilt  head.  Right  and  left  deflections  of  that 
controistick  command  pan  movements  of  -  175°  to  +  175°  to  the 
camera  positioning  device.  Y-axis  deflections  control  the 
camera  elevation.  Both  controlsticks  function  as  a  positioning 
zero-order  system.  As  such  the  camera  is  always  moved  back  to 
the  normal  position  when  the  controistick  is  undeflected.  This 
method  requires  a  precise  determination  of  the  normal  position 
with  respect  to  the  driving  task.  Normal  position  for  the 
experimental  setup  is  a  forward  looking  camera  attitude  with  a 
deflection  of  10°  .  With  the  optical  lens  used  the  TV  picture  is 
so  adjusted  that  the  hall  floor  limits  are  just  20%  under  the 
upper  end  of  the  TV  picture. 

Alternatively  two  controlsticks  are  used  that  are  installed 
in  the  operator's  aircraft  type  seat.  Using  these  operating 
conditions  the  operator's  forearms  are  placed  on  armrests.  The 
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control  sticks  are  in  reach  of  the  hands  and  fingers  and  car.  be 
deflected  with  hardly  any  forearm  movement  using  hand  and  finger 
movements .  The  third  control  device  available  is  a  6-degree-of 
freedom  sensor  ball  which  is  also  used  in  3D  manipulation  of 
computer  graphics  ob jeers  in  CAD  systems.  This  device  proved  to 
be  less  acceptable  mainly  due  to  the  fact  that  it  was  a  force 
sensing  device  with  practically  no  deflection.  Driving  quality 
with  it  was  poor  no  matter  which  of  the  degrees  of  freedom  was 
used  for  steering  and  acceleration. 

5.  STM80S  SIMULATION  QV  RORO^  SYSTEMS 

The  complexity  of  robot  systems  requires  detailed  analysis 
of  system  design  criteria,  the  comparison  of  system  concept^, 
and  the  detection  of  human  engineering  bottlenecks.  This  is 
especially  true  for  land  robots  in  a  complex  environment.  The 
development  of  prototype  systems  with  the  required 
specifications  and  even  a  whole  variety  of  hardware  platforms  is 
time  consuming  and  not  cost  effective.  The  simulation  of  such 
systems  is  therefore  required.  Simulator  studies  in  the 
laboratory  using  human  subjects  can  be  conducted  under  defined 
experimental  condisions .  These  conditions  can  be  exactly 
reproduced  and  evaluated  using  statistical  methods. 

In  the  SIMROS  project  phase  a  simulation  environment  is 
being  developed  which  supports  the  experimental  analysis  of 
interaction  between  the  operator  and  simulated  technical 
components  in  a  very  early  design  phase.  Such  simulation 
approaches  are  very  coarse  at  the  present  stage  and  support 
parameter  studies.  Further  program  phases  will  analyse  task 
structures,  function  allocation  between  robot  and  control 
station,  and  define  appropriate  process  and  mission  display- 
concepts  . 

Parallel  to  the  development  of  the  EROS  vehicle  a  real  time 
simulation  was  programmed.  The  robot  simulation  covers  a  vehicle 
of  similar  kinematic  and  dynamic  properties  as  the  real  car. 

This  model  can  be  operated  using  the  EROS  control  station.  The 
path  of  the  simulated  vehicle  can  be  monitored  on  a  digital  map 
display  much  like  the  procedure  using  a  physical  vehicle.  As 
such,  critical  mission  phases  are  tested  using  the  simulation 
before  executing  the  phases  using  real  equipment.  The  EROS  model 
can  also  be  used  to  investigate  the  system  performance 
concentrating  two  missions  in  a  single  operator  control  station. 

A  simulated  robot  view  was  also  developed  though  it  is  not 
yet  in  real  time  (Figure  6) .  The  path  of  the  simulated  vehicle 
can  be  monitored  on  the  map  display.  The  path  planning  and 
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navigator  systems  that  automatically  steer  the  vehicle  on  a 
preplanned  path  can  also  work  on  the  simulated  vehicle. 


Figure  6:  Simulated  view  from  a  simulated  robot  vehicle 


6.  HUMftM-MCai^.Iim^FACSJEOBuRQBOI.COMlBQL 

The  functions  available  at  the  robot  control  station  and  the 
information  processing  in  the  robot  itself  were  structured  into 
a  systematic  approach  for  system  information  input  and  system 
information  output.  The  system  input  is  shown  in  a  block  diagram 
in  Figure  7.  It  contains  blocks  of  functions  with  decreasing 
hierarchy  from  the  left  to  the  right  block  in  the  Figure.  In 
this  hierarchy  the  obstacle  acquisition  system  ROSE 
(Reconnaissance  of  Obstacles  and  Surveillance  in  unknown 
Environments)  provides  the  highest  degree  of  autonomy.  The 
teleoperational  mode  is  the  function  with  the  lowest  autonomy 
Outputs  of  the  ROSE  system  is  the  obstacle  map  in  a  former 
unknown  terrain.  Its  data  are  used  to  plan  paths  on  a  strategic 
level.  The  tactical  planner  finds  optimal  paths  in  the  terrain 
as  defined  by  the  strategic  planner.  The  navigator  uses  the 
waypoint  list  output  by  the  tactical  planner  taking  the  dynamic 
obstacle  avoidance  system  into  account.  This  systematic  approach 
supports  automatic  and  manual  operation  as  well  as  any  autonomy 
levels  in  between. 
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Figure  7:  EROS  system  input 

'he  corresponding  structure'  for  system  output  is  shown  in 
Figure  S. 


EROS  system 
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Figure  8:  EROS  system  output 


The  function  blocks  define  a 
unctions  that  are  accessible  thr 
inner  and  outer  system  variables 
variables  include  vehrcle  ceramet 
f  the  vehicle.  Outer  variables  a 
verview,  extrapolated  path  data. 


nodular  approach  of  system 
■ugh  the  robot  control  station, 
are  displayed.  The  inner 
n r 3  as  well  as  the  state  vector 
re  mission  data,  the  obstacle 
me  fused  sensor  informations. 
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The  arrows  indicate  that  the  console  operator  can  access  any 
data  in  an  interactive  approach. 


7  .  SFLF.CTFT  EXPERIMENTS  l'S~Nd  a  "TrRrp  VTFWTNd  SYSTEM 

Many  driving  experiments  have  already  been  conducted.  In  one 
experiment,  the  influence  of  a  stereo  TV  system  on  driving 
accuracy  was  measured.  The  operators  had  to  approach  a  part  of 
the  test  field  in  the  teleoperation  mode  that  was  unviable 
because  of  obstacles.  They  had  to  manoeuvre  as  closely  as 
possible  towards  the  obstacles  without  touching  them.  50%  of  the 
subjects  used  a  mono  display  first  and  later  switched  to  stereo. 
The  others  started  with  the  stereo  system  and  switched  to  the 
mono  system  in  the  second  part  of  the  experiment .  The  first  and 
second  path  of  each  of  the  driving  tours  were  evaluated  to  form 
a  separate  result  including  standard  deviation.  Thus  learning 
effects  can  be  easily  seen.  Figure  9  shows  selected  results. 


Figure 


Experimental  results  for  a  mono/ stereo  driving  task 


A  :  First  trial  group  of  experimental  condition 
B  :  Second  trial  group  of  experimental  condition 
.M.  :  Group  using  mono  display  first 
.3.  :  Group  using  stereo  display  first 
.  .M  :  Mono  experiments 
..S  :  Stereo  experiments 
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The  result  marked  AMM  shows  the  final  distance  to  the 
obstacles,  as  achieved  by  subjects  during  their  first  trials 
using  the  mono  display.  These  subjects  were  unfamiliar  with  the 
stereo  system,  ihe  _esult  BMM  shows  the  next  group  of  trials  of 
the  same  subjects,  it  also  shows  a  significantly  better  target 
approach  distance  compared  to  AMM.  The  significant  increase  in 
performance  is, due  to  learning. .  ASM  and  BSM  relate  to  the  trials 
of  subjects  who  used  the  stereo  sys-tem  first.  Their  performance 
w rs  significantly  better  but  showed  no  learning  effect . 

The  right  part  of  Figure  9  shows  the  results  of  stereo 
experiments.  A.gain  the  results  are  splitted  into  first  (A)  and 
last  half  of  the  trials  (B)  .  ASS  and  BSS  refer  to  subjects  who 
used  the  stereo  system  first.  These  data  show  a  significant 
learning  effect  again.  AMS  and  BMS  indicate  that  subjects,  who 
used  the  mono  display  first,  performed  worse  compared  to 
operators  with  stereo  experience.  They  show  no  further  learning 
effects. 

Consequently  stereo  vision  renders  better  results  which  is 
probably  due  to  the  development  of  a  more  precise  mental,  model 
of  the  environment . 
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Ongoing  experiments  defining  parameters  influencing 
teleoperation  of  a  vehicle  will  be  completed.  These  experiments 
include  investigation  about  the  performance  of  operators  using  a 
single  versus  multiple  controls  for  primary  driving  functions  as 
well  as  mission  oriented  control  tasks.  Displays  formats  of 
overlay  graphics  mixed  on  a  TV  channel  as  transmitted  from  the 
mobile  system  will  be  evaluated  in  an  experimental  series. 
Ergonomic  constraints  of  semiautcnomous  operation  including  the 
path  planner  and  navigator  will  be  studied.  Problems  of  function 
allocation  between  the  operator  and  the  robot  and  how  to  define 
tasks  for  the  robot  are  decisive  for  system  acceptance  and 
throughput  and  will  be  subject  to  human  engineering  research.. 
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1.0  ABSTRACT 

Over  the  past  decade,  the  Naval  Ocean  Systems  Center's  Hawaii 
Laboratory  has  engaged  in  a  program  to  develop  unmanned  ground  vehicles 
(UGVs)  that  have  been  delivered  to  the  United  States  Marine  Corps  for  field 
assessments  of  the  applicability  and  effectiveness  of  such  vehicles  for 
reconnaissance  and  combat  in  tactical  environments.  An  essential  component  of 
these  unmanned  ground  vehicles  is  a  visual  sensor  suite  and  helmet-mounted 
display  that  allows  an  operator  to  view  the  remote  scene  in  a  familiar,  natural 
fashion  well  enough  to  drive  the  UGV  safely  and  reliably  across  unfamiliar  off¬ 
road  terrain.  To  guide  the  development  of  this  mobility  sensor  system,  a  field 
testing  program  was  established  in  which  alternate  mobility  viewing  system 
options  were  objectively  compared  with  regard  to  their  impact  on  driving  under 
controlled  testing  conditions. 

This  report  describes  the  procedures  and  specific  tasks  used  in  making 
comparisons  across  the  various  viewing  system  options  tested.  The  experiment 
reported  here  was  run  with  two  groups  of  drivers;  1)  well-practiced  civilian 
personnel  who  were  tested  with  each  of  four  alternate  viewing  system  options, 
and  2)  unpracticed,  enlisted  Marine  personnel  who  volunteered  to  be  tested  with 
a  single  mobility  sensor  system  option  on  a  one-time  basis.  Specific  results  in 
terms  of  relative  driving  efficiencies  on  six  driving  are  reported  and  discussed 
with  respect  to  their  general  implications  for  design  of  the  man-machine  interface 
for  driving  remotely  operated  ground  vehicles. 


2.0  BACKGROUND 

The  Ground-Air  Telerobotics  (GATERS)  program  was  initiated  in  October 
1985  to  develop  teleoperated  vehicle  systems  for  military  applications.  Three 
TeleOperated  Vehicle  (TOV)  systems  were  delivered  to  the  U.S.  Marine  Corps 
for  field  assessments  of  their  operational  value  in  tactical  combat  environments. 
Observations  derived  from  initial  field  assessments  of  Teleoperated  Vehicle 
(TOV)  performance  and  their  implications  for  future  UGV  developments  are  the 


topic  of  another  paper  in  this  volume  of  proceedings  [1].  A  TOV  system  consists 
of  a  remotely-operated,  High  Mobility  Multi-purpose  Wheeled  Vehicle  (HMMWV), 
a  datalink,  and  a  control  interface  for  the  human  operator.  Several  alternative 
mission  modules  have  been  developed  for  the  TOV  to  support  a  variety  of 
observation  and  surveillance  missions  as  well  as  forward  target  designation  and 
light  weapons  engagement.  A  more  detailed  description  of  the  TOV  system 
including  its  control  system  architecture,  fiber  optic  datalink,  mobile  control 
station,  and  various  mission  modules  is  available  in  a  recent  report  [2].  In  this 
paper,  I  describe  results  from  a  series  of  controlled  vehicle  mobility  studies 
conducted  in  latel  986  -  early  1 987  at  an  outdoor  test  course  with  the  first 
operational  prototype  of  the  TOV. 

2.1  Viewing  Options. 

In  the  experimentation  described  here,  the  effects  of  four  different  viewing 
options  on  driving  performance  were  measured  and  compared.  The  four  viewing 
options  investigated  were:  1)  unobstructed  direct  view,  2)  direct  view  with  a  40° 
by  30°  field  of  view,  3)  monoscopic  helmet-mounted  display  ,  and  4) 
stereoscopic  helmet-mounted  display  Under  the  first  two  of  these  options,  drivers 
viewed  test  courses  directly;  that  is,  they  were  physically  present  on  the  test 
courses  in  the  vehicle  and  had  a  direct  line  of  sight  to  the  vehicle  and  courses 
while  driving.  With  the  remaining  two  viewing  options,  the  operators'  only  view  of 
the  vehicle  and  the  test  courses  was  provided  indirectly  by  means  of  a  video 
system. 

More  specifically,  under  the  "Direct  View"  (DV)  viewing  option,  the  vehicle 
operator  was  physically  present  in  the  driver’s  seat  of  the  HMMWV  with  a  direct, 
non-restricted,  "natural"  view  of  the  driving  course.  This  viewing  option 
established  a  "100%  telepresence"  performance  baseline  against  which  the 
other  three  viewing  options  could  be  compared.  Image  resolution,  contrast,  and 
color  sensitivity  were  limited  only  by  the  visual  capabilities  of  individual  drivers. 
Additionally,  the  DV  option  provided  "perfect"  head  motion  coupling  (i.e.,  no 
position  errors  or  motion  lags),  eye-head  coordination,  and  a  "natural"  1 :1 
spatial  correspondence  between  perceived  space  and  physical  space. 

Under  the  "40°  by  30°  Direct  View"  (DV  40  x  30)  viewing  option,  the 
driver's  view  of  the  test  site  was  identical  in  all  ways  to  that  of  the  DV  option, 
except  for  the  fact  that  his  peripheral  field  of  view  was  restricted.  The  central  40° 
by  30°  of  his  normal  binocular  field  was  visible.  Areas  outside  this  region  were 
occluded  and  not  visible.  Thus,  the  DV  40  x  30  viewing  option  provided  all  of  the 
advantages  of  the  DV  option,  but  with  a  field  of  view  restricted  to  that  available 
under  the  two  video  viewing  options  described  next. 

Under  the  two  video  view  options,  the  driver  was  also  physically  present 
in  the  driver's  seat  of  the  vehicle  while  driving  it,  but  his  only  view  of  the  test 
course  was  provided  indirectly,  by  means  of  a  video  system.  A  stereoscopic  pair 
of  cameras,  attached  to  the  top  of  his  helmet,  fed  a  pair  of  small  helmet-mounted 
CRTs,  each  of  which  was  visible  to  only  one  of  the  driver's  eyes.  Opaque  tape 
was  used  to  prevent  direct  view  of  the  driver's  surroundings.  The  helmet- 
mounted  display  (HMD)  used  in  this  experimentation  was  a  Honeywell 
Integrated  Head  Aimed  Display  Sighting  System  (IHADSS)  modified  to  provide 
two  separate  video  channels  for  stereo  viewing.  The  HMD  system  itself  weighed 
slightly  less  that  2.25  kg  and  afforded  its  wearer  a  40°  horizontal  by  30°  vertical, 
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fully  overlapped,  stereoscopic,  monochromatic  field  of  view.  A  stereoscopic 
(stereo)  pair  of  cameras  was  mounted  atop  the  HMD  with  optical  axes  separated 
by  65  mm  and  symmetrically  converged  on  a  point  approximately  8  meters 
ahead  of  the  TOV  front  bumper.  This  camera  pair,  it's  mounting  pla+e,  and 
attached  cables  added  approximately  1  Kg  to  the  overall  weight  burden  on  the 
operator's  head. 

Though  resolution  and  contrast  were  greatly  reduced,  and  color  contrast 
was  absent  from  the  video  images,  it  should  be  noted  that  both  these  video  view 
options  did  provide  the  driver  with  a  wealth  of  sensory  information  not  readily 
available  to  an  operator  controlling  a  UGV  from  a  remote  station.  Camera 
slewing  was  nearly  perfectly  matched  to  the  operator's  head  and  upper  body 
motions  with  only  very  slight  lags  and  distortions  in  the  displayed  imagery 
induced  by  the  video  scanning  rate.  And,  except  for  the  mismatch  between 
visual  and  vestibular  stimulation  caused  by  the  lack  of  1 :1  spatial 
correspondence  in  the  display  [  system  magnification  was  measured  to  be  ~.77] ; 
vestibular,  kinesthetic,  and  vibrational  information  was  generated  by  the  physical 
movement  of  the  vehicle  and  driver  through  the  courses.  This  non-visual  sensory 
information  provided  the  driver  with  immediate  feedback  regarding  body 
orientation  relative  to  the  vehicle  and  vehicle  dynamics  with  respect  to  the 
various  courses  run. 

Under  the  monoscopic  viewing  option  ("Mono  -  HMD"),  the  video  signal 
output  by  the  right  camera  was  split  and  fed  to  both  IHADSS  displays.  This 
provided  the  operator  with  a  two-dimensional,  "flat"  view  of  the  scene,  though 
many  visual  cues  to  depth  and  distance  were  discernable  within  the  "flat"  images 
provided.  [3] 

Under  the  "Stereo  -  HMD"  viewing  option,  images  from  the  left  and  right 
cameras  were  fed  to  their  corresponding  IHADSS  display  units.  This  viewing 
option  provided  the  driver  with  a  more  accurate  three-dimensional  view  of  the 
test  course  and  vehicle  during  testing. 

Under  all  four  viewing  options,  drivers  were  instructed  to  maintain  their 
upper  bodies  in  a  fixed,  upright  posture,  making  scanning  motions  only  with 
movements  of  their  heads.  This  was  done  to  make  scanning  of  the  scene 
roughly  equivalent  to  the  scanning  that  would  occur  under  teleoperation  with  a 
mechanical  pan  and  tilt  head.  Since  there  were  no  external  physical  constraints 
placed  on  drivers  other  than  a  lap  belt,  slight  deviations  from  the  instructed 
posture  were  observed.  However,  large  deviations  from  the  instructed  posture 
such  as  "leaning  out"  the  side  of  the  vehicle  to  better  view  obstacles  in  the  path 
of  the  vehicle  did  not  occur  during  data  collection. 

.  A  true  Remote  View  driving  condition  was  not  tested  in  the  initial  phase  of 
the  TOV  mobility  testing  program  reported  here.  However,  more  recent  studies 
that  are  beyond  the  scope  of  this  report  have  been  conducted  and  are  currently 
being  prepared  for  public  release  as  a  NOSC  Technical  Publication  [4], 

2.2  Experimental  Drivers 

Two  groups  of  drivers  participated  in  the  experiments.  An  "experienced" 
group,  consisted  of  four  civilian  personnel  (mean  age  =  43  years)  who  were 
licensed  HMMWV  operators  and  were  well-practiced  at  driving  all  of  the  test 
courses  used  in  the  experiment.  Under  both  Direct  View  and  Direct  Drive  Video 
View  conditions,  these  drivers  were  run  through  each  of  the  courses  a  minimum 
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of  ten  times  prior  to  experimental  data  collection.  A  graphical  analysis  of 
measures  taken  during  these  course  familiarization  runs  suggested  that  all 
"experienced"  drivers  had  reached  stable  levels  of  performance  on  all  measures 
taken  prior  to  completion  of  the  familiarization  runs.  Each  of  the  "experienced" 
drivers  was  run  under  all  viewing  conditions  tested  ( i.e.,  DV,  DV  40°  by  30°, 
Stereo  HMD,  Mono  HMD). 

A  second,  "inexperienced",  group  of  drivers  consisted  of  4  detachments  of 
4  Marines  each  ( mean  age  =  23  years).  Each  of  these  drivers  was  a  licensed 
HMMWV  operator  and  volunteered  to  serve  as  an  experimental  driver  during  a 
single  test  session.  All  drivers  from  both  the  "experienced"  and  "inexperienced" 
groups  had  normal  or  corrected-to-normal  visual  acuity  as  determined  by  a 
standard  Armed  Forces  Vision  Tester.  Prior  to  experimental  data  collection,  each 
of  the  "inexperienced"  operators  was  driven  around  the  entire  set  of  courses  by 
the  data  collector  and  instructed  in  the  specific  procedures  for  each  of  the  6 
courses  described  in  detail  below.  Immediately  prior  to  testing,  "inexperienced" 
drivers  were  allowed  one  practice  run  through  each  course  under  DV  conditions. 
These  instructional  runs  were  not  necessary  with  the  "experienced"  drivers  due 
to  their  high  degree  of  prior  familiarity  with  the  courses.  "Inexperienced"  drivers 
were  used  in  order  to  gain  an  appreciation  for  the  effects  of  learning  and 
experience  on  driving  performance  under  the  various  viewing  conditions  tested. 

2.3  Driving  Courses 

Six  driving  courses  were  used  in  an  effort  to  characterize  low-speed 
mobility  on  a  more  or  less  ideal,  paved  road  surface.  The  six  courses  used  were 
originally  selected  on  the  basis  of  a  factor  analysis  of  58  measures  of  low-speed 
maneuverability  [4]  conducted  at  the  University  of  Michigan's  highway  Safety 
Research  Institute  (HSRI).  This  battery  of  tests  was  developed  to  provide  a  cost- 
effective,  reliable,  reasonably  sensitive  and  comprehensive  metric  against  which 
vehicle  control  options  could  be  systematically  assessed  and  improved.  The 
testing  courses,  described  in  detail  below,  were  surveyed  and  marked  off  on  a 
level,  asphalt  covered  test  area  within  1  kilometer  of  NOSC-Hawaii's 
Teleoperator  Development  Center. 

Though  the  general  layout  of  courses  used  in  this  paper  was  described  in 
the  HSRI  report,  some  slight  modifications  to  course  configurations  and 
procedures  were  required  for  testing  with  TOV.  Bright  orange  76.2  cm  [  30  inch  ] 
tall  traffic  pylons  were  used  to  mark  off  all  course  boundaries.  In  certain  of  these 
traffic  pylons,  1 .82  meter  [  6  foot  ]  long,  2.54  cm  [1  inch  ]  diameter  white  rods 
were  inserted  to  make  them  visible  over  the  high  hood  and  rear  flatbed  of  the 
TOV.  Order  of  administration  for  the  6  courses  was  identical  for  all  drivers  on  all 
days  of  testing  and  followed  the  order  in  which  they  are  described  below.  For  all 
courses,  verbal  instructions  were  given  which  emphasized  the  importance  of 
accurate,  error-free  driving  and  de-emphasized  the  importance  of  speed. 

2.3.1  Course  1.  Right  Angle  Turn-  IN. 

The  first  course  run  during  each  test  session,  the  Right  Angle  Turn-  IN,  is 
diagrammed  in  the  left  panel  of  Figure  1.  The  course  is  configured  as  a  3.35 
meter  (1 1  feet)  wide  right  angle  parking  space  connected  to  a  perpendicular  5.8 
meter  (14  feet)  wide  access  lane.  The  driver's  task  was  to  start  at  one  end  of  the 
access  lane  and  make  a  controlled  right-angle  turn  pulling  as  far  into  the  parking 
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space  as  possible  without  touching  any  of  the  cones  defining  the  course  or 
touching/toppling  the  stop  cone  centered  at  the  end  of  the  parking  space  with  the 
front  bumper  of  the  vehicle.  Runs  were  scored  for  elapsed  time,  number  of  cones 
touched  or  toppled  during  the  maneuver,  and  accuracy  of  position  relative  to  the 
parking  space  endpoint.  The  Right  Angle  Turn-IN  was  executed  a  total  of  6  times 
per  session,  3  times  each  from  right  and  left  START  positions. 


2.3.2  Course  2.  Right  Angie  Turn-  OUT. 

As  the  right  panel  in  Figurel  illustrates,  starting  from  the  position  in  which 
the  vehicle  rested  following  the  previous  Right  Angle  Turn-IN  run,  the  vehicle 
was  reversed  out  of  the  parking  space  and  across  the  access  lane.  It  was  then 
put  into  forward  gear  and  driven  out  of  the  access  lane  in  the  same  direction 
from  which  it  had  been  driven  in.  Runs  were  scored  for  elapsed  time  and  number 
of  cones  touched  or  toppled  during  the  maneuver.  As  with  the  Right  Angle  Turn- 
IN  procedure,  6  runs  were  measured  per  session;  3  from  each  START  position. 

2.3.3  Course  3.  Figure-8. 

The  next  course  required  an  operator  to  negotiate  a  figure-8  pattern 
through  a  sparse  set  of  cone  gates  (see  Figure  2).  Spacing  between  cones 
comprising  the  gates  was  widened  from  the  original  HSRI  specification  in  order 
to  accommodate  the  HMMWV  turning  radius.  A  single  experimental  run  of  the 
course  consisted  of  three  consecutive  circuits  through  the  figure-8  pattern.  Runs 
were  scored  for  elapsed  time  and  number  of  cones  touched  or  toppled.  Two  runs 
were  measured  per  session.  • 


FIGURE  3.  The  Small  Radius  Circle  Courses. 
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2.3.4  Course  4.  Small  Radius  Circle. 

The  Small  Radius  Circle  course  is  depicted  in  the  left  panel  of  Figure  3. 
The  START  position  for  this  course  was  30.48  meters  [100  feet]  from  the  first 
cone  gate.  During  a  testing  session  operators  drove  the  course  twice  from  each 
of  the  START  positions  depicted  in  the  figure.  Runs  were  scored  for  elapsed  time 
and  number  of  cones  defining  the  curved  alleyway  touched  or  toppled. 
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2.3.5  Course  5.  Smali  Radius  Circle  with  Stop. 

The  Small  Radius  Circle  with  Stop  course  is  depicted  in  the  right  panel  of 
Figure  3.  The  layout  of  this  course  identical  to  that  of  the  Small  Radius  Circle 
with  the  addition  of  a  stop  cone  at  the  apex  of  the  arc.  Drivers  were  instructed  to 
position  the  front  of  the  vehicle  as  close  to  the  stop  cone  as  possible  without 
touching  it.  During  a  test  session  the  course  was  run  twice  from  each  of  the  two 
START  positions  depicted  in  Figure  3.  Runs  were  scored  for  elapsed  time, 
number  of  cones  defining  the  curved  alleyway  touched  or  toppled,  and  accuracy 
of  position  relative  to  the  stop  cone  endpoint. 

2.3.6  Course  6.  Gymkhana. 

The  gymkhana  course  was  a  large,  oval-shaped  slalom  course  depicted 
in  Figure  4.  Cone  separation  for  each  of  the  1 0  cone  gates  was  2.75  meters  [  9 
feet  ].  Three  separate  runs  through  the  course  were  undertaken  in  each  session. 
Runs  were  scored  for  elapsed  time  and  number  of  cones  touched  or  toppled. 


3.0  RESULTS 

3.1  Statistical  Procedures  Used 

Performance  metrics  from  each  of  the  driving  courses  were  compiled  and 
subjected  to  separate  analyses  of  variance  (ANOVAs)  with  comparisons  across 
the  four  viewing  options  (DV,  DV  40  x  30,  Stereo-HMD,  and  Mono-HMD)  being 
the  effect  of  paramount  interest  in  each  analysis.  A  Type-1  error  level  of  .05  for 
statistical  significance  was  set  prior  to  analysis.  Repeated-measures  ANOVAs 
were  run  on  the  results  from  the  "experienced"  drivers  and  between-groups 
ANOVAs  were  run  on  the  measures  from  "inexperienced"  drivers.  Findings  are 
presented  below  from  all  courses  run  across  four  design  topic  areas. 
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3.2  Direct  Viewing  Versus  HMD  Viewing 

Nol'  surprisingly,  all  statistically  significant  differences  that  were  found 
favored  the  combined  Direct  View  ( i.e.,  DV  &  DV  40  x  30)  conditions  over  the 
combined  Video  View  ( i.e.,  Mono  -  HMD  &  Stereo  -  HMD)  conditions.  The 
overall  pattern  of  results  that  emerged  from  mean  comparisons  subsequent  to 
the  ANOVAs  is  presented  in  Table  1 . 


TABLE  1.  Summary  of  Statistically  Significant  Advantages  for 
Combined  Direct  View  over  Combined  Video  View 
Conditions. 


"Exper 

Dm 

Time 

enced" 

/ers 

Errors 

“Inexperie 

Driver 

Time 

need" 

s 

Errors 

Right-Angle 

Turn-IN 

-60% 

faster*" 

■mm 

n.s.d. 

n.s.d. 

Right-Angle 

Turn  -  OUT 

-34% 

faster** 

n.s.d. 

n.s.d. 

n.s.d. 

Figure  -8 

-31% 

faster* 

n.s.d. 

n.s.d. 

-65% 

fewer** 

Small  Radius 
Circle 

-25% 

faster* 

n.s.d. 

n.s.d. 

n.s.d. 

Small  Radius 
Circle- Stoo 

-13% 

faster* 

n.s.d. 

n.s.d. 

n.s.d. 

Gymkhana 

-15% 

faster** 

n.s.d. 

n.s.d. 

-50% 

fewer* 

Key: 

*  Type-1  error  probability  <  .05 
**  Type-1  error  probability  <  .01 
n.s.d.  =  no  significant  difference  found 


In  summary,  statistically  significant  differences  in  driving  performance 
measures  were  found  for  each  of  the  six  measures  taken  in  the  mobility  test 
battery.  All  of  these  differences  showed  performance  under  the  direct  viewing 
conditions  to  be  superior  to  that  under  the  video  viewing  conditions  tested. 
Differences  were  notably  inconsistent  between  the  two  driver 
groups."lnexperienced"  drivers  produced  higher  rates  of  errors  on  error 
measures  with  no  significant  differences  for  the  time  measures.  The  pattern  of 
results  suggests  that  "inexperienced"  drivers  attempted  to  drive  the  vehicle  at 
equal  speeds  under  direct  view  and  video  view  options.  "Experienced"  drivers 
drove  the  time-scored  courses  faster  under  direct  view  conditions  while 
maintaining  consistent  error  rates  across  the  viewing  options  tested.  As  one 
example  of  significant  performance  effects  revealed  by  the  set  of  analyses 
conducted,  the  effect  of  viewing  system  option  on  Gymkhana  course  times  for  the 
"experienced"  group  of  drivers  is  graphed  in  Figure  5.  While  maintaining 
essentially  equivalent  error  rates  for  all  conditions  tested,  the  performance 
graphed  in  the  figure  is  nearly  identical  for  the  two  direct  view  conditions, 
whereas  times  are  slower  under  the  Stereo-HMD  option  and  even  slower  under 
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the  Mono-HMD  option.  Overall,  driving  time  under  the  direct  view  options  was 
approximately  15%  faster  than  driving  time  under  video  view  options. 


FIGURE  5.  Effect  of  Viewing  Option  on  Gymkhana  Course  Times  fpr 
the  Experienced  Group  of  Drivers.  '■< 


"EXPERIENCED"  DRIVERS 
Gymkhana  Course  Times 


Viewing  Option 

Note:  the  error  bars  graphed  in  this  figure  and  subsequent  figures  represent  standard  errors  defining  the 
upper  bound  of  the  95%  confidence  interval. 

3.3  Unobstructed  Versus  Masked  Video  Viewing 

Given  the  richly  detailed  visual  information  provided  under  both  direct 
view  options,  no  reliable  performance  difference  was  found  between  the 
unoccluded  DV  option  and  the  peripherally  occluded  DV  40  x  30  option.  Though 
error  rates  were  consistently  observed  to  be  slightly  elevated  for  the  DV  40  x  30 
option  versus  the  DV  option  for  both  driver  groups  on  ail  driving  courses  tested, 
none  of  these  error  rate  differences  were  found  to  be  statistically  significant.  This 
outcome  of  the  analysis  suggests  that  if  sufficient  image  resolution,  contrast, 
color,  head  motion  coupling,  and  accurate  feedback  of  vehicle  dynamics  are 
provided  to  a  driver,  a  40°  by  30°,  1 :1  field  of  view  is  sufficiently  wide  enough  for 
low-speed  mobility  within  the  scope  of  fundamental  driving  tasks  tested  in  this 
study.  However,  the  consistency  of  the  pattern  of  slightly  elevated  error  rates  for 
the  DV  40  x  30  option  versus  the  DV  option  does  suggest  that  some  capability  is 
lost  by  restricting  a  driver's  peripheral  field  of  view.  Whether  this  effect  is  stronger 
for  driving  over  more  challenging  terrain  remains  to  be  determined  and  should 
be  investigated. 

3.4  Stereoscopic  Versus  Monoscopic  Video  Viewing 

Measures  were  taken  with  the  same  IHADSS  helmet-mounted  display  on 
identical  driving  courses  under  two  viewing  options;  monoscopic  and 
stereoscopic.  An  earlier,  preliminary  analysis  of  this  data  [4]  revealed  no 
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significant  differences  between  the  Mono-HMD  and  Stereo-HMD  viewing 
-options  on  any  of  the  courses  tested  for  either  operator  group.  The  more  refined 
analysis  conducted  for  this  paper  in  which  the  two  driver  groups,  were  analyzed 
separately,  contradicted  the  findings  of  the  previous  analysis  by  revealing 
statistically  significant,  but  modest,  performance  advantages  for  use  of  a 
stereoscopic  display  on  several  of  the  courses  tested.  Table  2  summarizes  these 
significant  effects. 

TABLE  2.  Summary  of  Statistically  Significant  Advantages  for 
^  Stereo-HMD  Over  Mono-HMD  Viewing  Options. 


"Experienced" 

Drivers 

"Inexperienced" 

Drivers 

Time 

Errors 

Time 

Errors 

Right-Angle 

Turn  -  IN 

n.s.d. 

n.s.d. 

-36% 

slower* 

n.s.d. 

Figure-8 

n.s.d. 

n.s.d. 

-30% 

slower* 

n.s.d. 

Small  Radius 
Circle 

-13% 

faster* 

n.s.d. 

n.s.d. 

-61% 

fewer* 

Small  Radius 
Circle  -  Stoo 

-15% 

slower* 

n.s.d. 

n.s.d. 

n.s.d. 

Gymkhana 

-06% 

faster* 

n.s.d. 

n.s.d. 

n.s.d 

Key: 

*  Type-1  error  probability  <  .05 

n.s.d.  =  no  significant  difference  found  between  Stereo-HMD  and  Mono-HMD 


With  regard  to  vehicle  speeds,  it  appears  that  the  Stereo-HMD  option 
allows  "experienced"  drivers  to  maintain  faster  speeds  than  the  Mono-HMD  on 
several  of  the  courses  without  a  corresponding  penalty  in  error  rates.  This 
pattern  is  apparent  in  Figure  6,  a  graph  of  the  effect  of  viewing  option  on  course 
times  on  the  Small  Radius  Circle  course  for  the  "experienced"  group  of  drive's.  A 
possible  benefit  of  Stereo-HMD  for  the  "inexperienced"  group  may  be  that  it 
imparts  a  more  accurate  sense  of  space,  and  that  this,  in  turn,  motivates  them  to 
slow  down  and  make  fewer  errors  when  given  a  video  view  of  a  relatively 
unfamiliar  driving  situation.  This  conclusion  is  suggested  by  the  patterns  of 
performance  graphed  in  Figures  7  and  8.  The  pattern  of  results  graphed  in 
Figure  7  shows  that  "inexperienced"  drivers  tended  to  drive  significantly  faster 
under  the  Mono-HMD  versus  the  Stereo-HMD  viewing  option.  A  similar  pattern 
(not  graphed)  was  found  for  "inexperienced"  drivers  on  the  Figure-8  course.The 
"staircase"  pattern  of  results  graphed  in  Figure  8  was  apparent  in  the  data  from  a 
majority  of  courses  from  both  driver  groups,  though  it  only  reached  statistical 
significance  in  one  instance  (i.e„  Small  Radius  Circle  course  errors  for  the 
"inexperienced"  group)  probably  owing  to  the  small  size  of  the  sample  of  drivers 
tested 
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FIGURE  6.  Effect  of  Viewing  Option  on  Small  Radius  Circle  Times 
for  the  "Experienced"  Group  of  Drivers. 
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FIGURE  7.  Effect  of  Viewing  Option  on  Right  Angle  IN  Course 
Times  for  the  "Inexperienced"  Group  of  Drivers. 
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FIGURE  8.  Effect  of  Viewing  Option  on  Right  Angie  IN  Course 
Errors  for  the  "Inexperienced"  Group  of  Drivers. 
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Viewing  Condition 


In  attempting  to  generalize  these  findings  of  a  stereoscopic  advantage  to 
more  rigorous  and  challenging  UGV  driving  conditions,  one  must  keep  in  mind 
that  past  research  and  hands-on  field  experience  has  shown  that  the 
performance  advantages  which  stereoscopic  imagery  provides  are  most 
pronounced  in  unfamiliar,  visually  cluttered,  and  visually  degraded  images. 
Stereo  imagery  is  also  known  to  be  useful  in  judging  the  relative  distances  and 
orientations  of  objects  and  terrain  surface  features  -  all  of  which  are  invaluable  to 
an  operator  when  evaluating  the  composition  and  topography  of  terrain  before 
attempting  to  traverse  it.  The  results  of  this  study  are  strongly  suggestive  of 
potential  performance  advantages  to  be  derived  by  using  stereoscopic  imagery 
in  UGV  display  systems.  A  more  relevant,  systematic,  controlled  comparison  of 
UGV  performance  with  stereoscopic  and  monoscopic  imagery  is  recommended. 


3.5  "Experienced"  Versus  "Inexperienced"  Drivers 

The  observed  effect  of  operator  experience  on  driving  the  TOV  through 
the  test  courses  used  in  this  experiment  can  be  summarized  quite  simply.  The 
"experienced"  group  tended  to  drive  more  conservatively  than  the 
"inexperienced"  group.  Under  video  viewing  conditions,  "experienced"  drivers 
slowed  down  and  made  fewer  errors  on  all  the  driving  courses  tested  than  their 
"inexperienced"  counterparts.  The  "inexperienced"  group  tended  to  drive  the 
TOV  at  speeds  approximating  those  achieved  under  the  two  direct  viewing 
conditions,  but  in  doing  so  they  committed  many  more  errors  than  the 
"experienced"  group.  Perhaps  the  observed  difference  in  performance  is  really 
just  a  difference  in  risk-taking  between  the  two  groups.  The  "experienced"  group 
was  considerably  older  and  more  technically  astute  than  the  "inexperienced" 


group.  Overall,  this  pattern  of  results  suggests  that,  given  several  hours  of 
experience  in  driving  the  TOV,  drivers  became  more  cautious  and  lowered  their 
driving  speed  to  better  correspond  to  their  degraded  view  of  the  courses. 

V  '  .  ■ 

4.0  IMPLICATIONS  FOR  UGV  DEVELOPMENT  &  DESIGN 

Even  while  driving  under  the  simplified,  relatively  undemanding,  and 
benign  course  conditions  investigated  in  this  early  study,  a  large  performance 
gap  was  documented  between  direct  view  and  video  view  driving.  These 
observed  differences  in  performance  are  directly  attributable  to  parameters  of 
the  viewing  systems  used  to  view  the  driving  situation  since  other  aspects  of  the 
driving  situation  ( i.e.,  steering  controls  and  forces  fed  back  through  them  to  the 
driver,  acoustic  feedback,  the  dynamics  of  sensor  aiming,  and  body  motion 
senses)  were  generally  equivalent  under  all  viewing  options  tested.  The  findings 
suggest  that  considerable  thought  and  effort  will  be  required  to  devise  UGV 
viewing  system  hardware  that  will  overcome  the  technological  limitations 
inherent  in  available  video  displays  and  provide  a  reasonable  approximation  to 
the  level  of  performance  achievable  under  direct  drive  conditions.  Several 
obvious  discrepancies  between  direct  viewing  and  video  viewing  merit  further 
systematic  investigation.  These  include  system  magnification,  spatial  resolution, 
and  image  contrast.  The  best  way  to  gauge  the  effects  of  these  and  other 
viewing  system  parameters  on  UGV  driving  is  by  systematic,  controlled 
comparisons  of  objective  performance  measures. 

Field  of  view  is  a  fundamental  parameter  of  any  UGV  viewing  system. 
Results  of  this  investigation  suggest  that  a  40°  horizontal  by  30°  vertical  field  of 
view  with  high  fidelity  head  movement  coupling  of  camera  aim  can  provide  a 
close  approximation  to  the  level  of  driving  performance  achievable  with  a  full, 
unoccluded  field  of  view  for  the  type  of  low-speed  mobility  measured  here. 
However,  it  should  be  noted  that  this  conclusion  is  here  only  validated  for  a 
situation  in  which  other  important  system  parameters  such  as  image  resolution, 
image  contrast,  and  system  magnification  are  equivalent  to  those  under  direct 
viewing  conditions.  An  objective  study  of  the  effects  of  a  fixed  sensor  field  of  view 
on  remote  driving  performance  is  currently  underway  at  the  US  Army's  Human 
Engineering  Laboratory.  The  results  of  this  study  should  hold  important 
implications  for  selection  of  an  appropriate  field  of  view  for  future  UGV  systems. 

Use  of  an  easily  implemented  stereoscopic  video  display  improved 
driving  performance  significantly,  though  modestly,  on  several  of  the  courses 
tested.  By  providing  the  driver  with  a  more  accurate  internal  spatial 
representation  of  the  driving  course  and  his  vehicle's  movements  within  that 
course,  the  driver  was  perhaps  able  to  make  better  decisions  regarding  speed 
and  steering  of  the  vehicle.  For  drivers  who  are  highly  experienced  at  performing 
a  particular  driving  task,  the  additional  information  may  allow  them  to  drive  faster 
while  maintaining  an  acceptable,  stable  risk  of  erring.  For  drivers  unpracticed  at 
performing  the  particular  driving  task  demanded  by  a  driving  course,  the 
additional  information  provided  by  a  stereoscopic  display  may  increase  their 
awareness  of  the  limitations  of  their  view  and  control  over  the  vehicle  and  may 
subsequently  encourage  them  to  reduce  speeds  and  drive  more  conservatively 
in  unfamiliar  situations. 


The  results  presented  in  this  report  point  very  consistently  to  the  general 
conclusion  that  providing  the  UGV  driver  with  a  display  that  more  closely 
approximates  the  "full  telepresence"  viewing  condition  is  a  successful  approach 
to  improving  driving  performance  as  measured  by  systematic,  objective  test 
procedures. 

5.0  ACKNOWLEDGEMENTS 

This  paper  would  not  have  been  possible  without  contributions  from  many 
individuals.  I  am  particularly  indebted  to  David  Smith  &  Brian  Nobunaga  of 
NOSC-Kawaii,  Dr.  Robert  E.  Cole  of  the  University  of  Hawaii,  and  Dr.  Ross 
Pepper  at  California  Polytechnic  Institute,  who  took  many  hours  from  their  busy 
work  schedules  to  serve  as  experienced  drivers.  I  am  also  indebted  to  Lt.  Kevin 
Monahan  and  the  many  enlisted  men  of  the  Third  Division  TOW  platoon  of  the 
First  Marine  Amphibious  Brigade  at  the  Kaneohe  Marine  Corps  Air  Station  who 
volunteered  their  valuable  time  and  efforts  to  serve  as  "inexperienced"  drivers. 
Clifford  Horikawa,  Rowland  Nishijo,  Alan  Umeda,  and  Derrick  Kusuda,  ail  of 
NOSC-Hawaii,  provided  excellent  technical  assistance.  Walter  Carvalho, 

George  Iwamoto,  and  Bart  Bunney,  also  of  NOSC-Hawaii,  lent  their  expertise  in 
surveying  and  marking  off  the  fundamental  mobility  testing  course.  Mr.  Fred  Heil 
of  SEACO,  Inc.  and  Clyde  Young  and  Kawehi  Lum,  engineering  students  from 
the  University  of  Hawaii,  served  as  on-site  safety  supervisors  and  data 
gatherers. 


6.0  REFERENCES 

[1]  Umeda,  A.Y.,  Martin,  S.W.,  and  Merritt,  J.O.  "Remote  vision  systems  for 
teleoperated  unmanned  ground  vehicles".  Proceedings  of  the  NATO 
Defence  Research  Group's  31st  Seminar  on  Robotics  in  the  Air-Land 
Battlefield.  Paris,  France:  6-8  March  1990. 

[2]  Aviles,  W.A.,  Hughes,  T.W.,  Everett,  H.R.,  Umeda,  A.Y.,  Martin,  S.W., 
Koyamatsu,  A.H.,  Solorzano,  M.R.,  Laird,  R.T.,  and  MacArthur,  S.P. 

"Issues  in  mobile  robotics:  The  Unmanned  Ground  Vehicle  Program 
TeleOperated  Vehicle".  Proceedings  of  the  Society  of  Photo-Optical 
Instrumentation  Engineers  Symposium  on  Advances  in  Intelligent 
Systems.  Vol.  1388,  Boston,  MA,  5-9  November  1990. 

[3]  Olsen,  P.L.,  Butler,  B.P.,  Burgess,  W.T.,  and  Sivak,  M.  "Toward  the 
development  of  a  comprehensive  driving  test".  Ann  Arbor,  Ml:  University 
of  Michigan  Highway  Safety  Research  Institute  Report  UM-HSRI-82-4, 
February  1982. 

[4]  Spain,  E.H.  Assessments  of  maneuverability  with  the  TeleOperated 
Vehicle  (TOV).  Proceedings  of  the  14th  Annual  Association  for  Unmanned 
Vehicle  Systems  Technical  Symposium  and  Exhibit.  Washington,  D.C., 
19-21  July  1987. 


CIO 


EEliSZIGNEMENTS  3 I 2 L IC GEA? HINGIS 


1.  Reference  cu  casoinataira: 

2.  Auer a  Reference: 

i.  Reference  s'origina: 

4.  Classification  de  sacurita:  i 

UNCLASSIFIED/UNLIMITED  i 

5.  Date:  6,  Nbre  ca  pages: 

Q1/19A3 _ _ 2 _ 

7.  litre  (NSC): 

Interactions  homme / machine  pour  les  robots  mobiles  militaires  . 


Scumis  a:  ... 

Serainaire  du  GKD  :  Robotique  du  champ  de  bataille  (6  au  8/03/1991) 


S.  Autaur;  s)/Ecitaur(s) : 

G.  CLEMENT  -  Ph.  GRAVE Z  -  R.  CADE 


10.  Acrassa  ca  T  auteur/ aditsur: 

CEA/DTA/UR 
BP  6 

92265  Fontenay-aux-Roses  Cedex 
France 


11.  ?c ‘ r.t  sa  -cr.tact  a  1 1 0*AN: 

Section  Racrarora  pour  la  Cafsr.sa 
Quart: ar  Genera!  C3  i  'CTAN 
3-1110  Sruxailas 
Belgique 


12.  0i3zr-.sus:sr.: 


Approved  for  public  release.  Distributi«. 
of  this  document  is  unlimited  and  is  not 
controlled  by  NATO  policies  or  security 
regulations. 


13.  Mots  cles/Descripteurs  : 

Interface  homme/machine,  Robotique  mobile.  Teleoperation. 


14.  Resume:  *  '  . 

L' interaction  homm e-machine  en  robotique  militaire  prend  place  a| 
plusieurs  niveaux  selon  le  degre  d'autonomie  du  systeme  commande.  Les 
informations  echangees  peuvent  etre  de  type  abstrait  et  discontinu  (ou 
symboliques)  ou  bien  de  type  continu  (ou  analogique)  dans  le  cas  d'une 
commande  en  teleoperation. 

La  qualite  de  1' interface  hnmme-robot  condi tionne  les  performance 
finales  du  systeoe.  L'environnement  militaire  presente  de  nombreuses  j 
difficulty  pour  un  systSoe  robotise,  et  notamoent  la  presence  des  menafc 
enneoies  intelligentes .  L’ intervention  huaaine  eat  necessaire  afin  de 

pouvoir  lea  suroonter .  ,  . 

Les  difficultes  de  conception  proviennent  d'une  part  des  limitatji 
de  performances  des  organes  "d'entrees-sorties"  de  1' homme  et  d* autre 
part  de  la  grande  difference  des  modes  de  "raisonnement"  entre  1' homme 
et  la  machine.  , 

La  connaissance  actuelle  est  aases  eopirique ,  un  approfondissemon 
theorique  et  experimental  est  necessaire. 


s 

es 

ons 


C10.1  AC/243-TP/3 

INTERACTIONS  HOMME/MACHINE 
POUR  LES.  ROBOTS  MOBILES  MILITAIRES 


G.CLEMENT*,  P.GRAVEZ*,  R.CADE* 

*  CEA/DTA/UR  Fontenay  aux  roses  Cedex  ,BP6  92265,  FRANCE. 
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1.  INTRODUCTION: 

Le  probleme  de  la  conception  et  de  ^optimisation  de  T  interface  homme-machine  dans 
les  applications  militaires,  n’est  pas  specifique  a  la  robotique.  On  le  rencontre  chaque  fois 
qu’un  systeme  comporte  des  automarismes  et  lorsque  un  flux  important  d’infoimations  est 
echange  avec  les  operateurs.  La  maitrise  des  avions  de  combat  modemes  par  exemple  exige 
une  concentration  tres  importante  des  pilotes,  de  nombreuses  etudes  sont  effectuees  afin  de 
reduire  cette  charge  de  travail. 

L’approche  robotique  place  neanmoins  le  probleme  de  l’interface  homme/machine  au 
premier  plan,  En  effet  un  engin  robotise  peut  se  definir  comme  1’aboutissement  de  la 
demarche  conduisant  au  developpement  d’axmes  et  de  systemes  automatiques. 

En  premiere  analyse  une  contradiction  peut  apparaitre:  plus  un  systeme  est  automatise 
moins  il  est  en  contact  avec  les  operateurs  humains  pendant  la  phase  operationnelle  (ce  qui 
constitue  un  des  objectifs  de  rautomatisation),  aussi  l’interaction  homme-machine  prend-elle 
moins  d’imponance  dans  ce  cas?  Cette  contradiction  n’est  qu’apparente,  en  effet: 

-  Un  systeme  robotique  entre  toujours  en  contact  avec  un  operateur  humain  au  corns 
de  son  cycle  de  fonctionnement,  ne  serait  ce  qu’une  seule  fois  lors  de  la  specification  initiate 
de  sa  mission.  Une  relation  homme-machine  s’&abiit  done  h  cette  occasion. 

-  Les  systemes  disposant  d’un  haut  degre  d’automaticite  echangent  des  informations 
de  contenu  semantique  eleve  avec  les  operateurs.  Or  plus  le  niveau  d’abstraction  de  la 
commande  est  eleve  plus  les  consequences  d’une  erreur  de  specification  de  la  tache  peuvent 
etre  graves  (ce  ne  sont  plus  seulement  des  erreurs  de  trajectoire).  Si  I’information  est  plus 
legere  quantitativement  elle  s’alourdit  qualitativement,  I’interface  homme-systeme  doit  etre 
conqu  de  faqon  h  minimiser  les  possibilites  d’erreurs. 

-  L’environnetnent  du  champ  de  bataille  est  complexe,  tres  hostile  et  difficile  h 
apprehender  par  un  systems  autonome.  En  particulier  il  presente  une  coroposante  nouvelle 
pour  la  robotique  (qui  est  specifique  i  Fapplicatian  militaire),  representee  par  la  menace 
qu’exerce  1’ennemi  sur  le  robot.  Cette  menace  est  agressive,  elle  ne  peut  se  reduire  a  un 
obstacle  classique  de  nature  geometrique  par  exemple  (un  relief  du  terrain  qu’il  sufflt  de 
contoumer).  De  plus  cette  menace  est  "intelligence",  evolutive  et  imprevisible  ce  qui  contribue 
a  augraemer  la  destructuration  de  I'envixonneraent  du  robot  qui  ne  pouira  reagir  efficaccment 
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sans  l’aide  d’une  "intelligence"  humaine.  L’assistance  d’un  (ou  de  plusieurs)  operateur(s)  est 
done  souvent  indispensable  pour  mener  a  bien  une  mission  robotique  militaire. 

Les  difficultes  de  conception  de  l’interface  homme-systeme  apparaissent  a  deux 
niveaux: 

-  Au  niveau  de  l’interface  physique  car  l’homme  presente  une  organisation  tres 
paraculiere  de  ses  "entrees-sorties"  ainsi  que  des  limitations  qui  sont  differentes  de  celles  de 
la  machine. 

-  Au  niveau  decisionnel,  les  representations  internes  de  1’environnement  et  de  la  tache 
sur  lesquelles  sont  basees  les  reactions  de  l’homme  et  du  robot  sont  tres  differentes.  Le  robot 
a  besoin  de  donnees  precises  et  bien  structurees,  toutes  les  ressources  doivent  etre  declaxees 
de  fagon  explicite.  L’homme  se  contente  d’informations  plus  floues,  il  dispose  d’une  base  de 
connaissances  tres  developpee  et  de  capacites  d’inferences  incomparables  (il  est  capable  par 
exemple  d’exploiter  des  images  video  de  tres  mauvaise  qualite  par  reconstruction  mentale  du 
contexte,  la  ou  le  robot  ne  peut  extraire  aucune  information  pertiiiente). 

2.  ACTIVITE  HOMME/MACHINE 

On  peut  decomposer  l’activite  des  operateurs  en  suivant  la  segmentation  habituelle  de 
la  commande  d’un  robot  mobile.  Les  trois  niveaux  foncdonnels,  dans  l’ordre  decroissant  du 
traitement  semantique  sont: 

-  Le  niveau  navigation. 

-  Le  niveau  pilotage. 

-  Le  niveau  guidage. 

Le  dialogue  homme-machine  peut  intervenir  a  chacun  de  ces  niveaux  selon  le  type  de 
systeme  commande.  L’interacrion  peut  aussi  prendre  place  simultanement  a  plusieurs  niveaux, 
par  exemple  lorsqu’un  premier  operateur  pilote  directement  l’engin  pendant  qu’un  second 
operateur  surveille  les  menaces  adverses  et  assiste  le  premier  en  suivant  le  deroulement  global 
de  la  mission. 

La  conception  de  i’interface  depend  directement  de  la  nature  des  informations 
echangees  et  de  1’activite  correspondante  demandee  a  l’operateur,  ce  que  1’on  detaille  ci-apres. 

2.1  NAVIGATION 

Les  activites  prenant  place  au  niveau  navigation  concement  tout  d’abord  la  definition 
de  I’objectif.  Il  est  specifie  en  termes  foncdonnels  et  topologiques  ce  qui  comprend: 

>  La  definition  d’un  trajet  general  1  suivre  comme  par  exemple:  suivre  la  route 

jusqu'au  croisement,  toumer  a  droite  apres  I’arbre,  avancer  rapidement  jusqu’au 

sommet  de  la  coiline  ... 
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-  La  definition  des  taches  a  realiser  sur  ce  trajet,  ex:  inspection  panoramique  IR 
pendant  le  parcours,  relever  la  position  des  points  chauds,  tirer  sur  les  objectifs 
detectes ... 

-  La  precision  d’attributs  et  de  contextes  comme:  des  informations  precisant  la  nature 
des  objets  que  le  robot  est  susceptible  de  rencontrer  dans  son  environnement  (zones 
ennemies  connues,  zones  interdites,  zones  infranchissables,  amers  remarquables ...). 
Les  comportements  a  adopter  dans  differentes  configurations  et  notamment  dans  les 
cas  critiques  (ex:  perte  de  communications  radio,  panne  ou  destruction  partielle  d’un 
sous  systeme  ...). 

Cette  premiere  activite  prend  place  principalement  hors  ligne,  pendant  la  phase  de 
preparation  de  la  mission.  Elle  peut  etre  assistee  par  un  dispositif  permettant  la  simulation  si 
un  modele  de  l’environnement  suffisamment  representatif  est  disponible. 

Le  support  utilise  pour  effectuer  cette  "programmation"  est  generalement  constitue  par 
une  cartographic  plane  de  1’ environnement  sur  laquelle  la  definition  de  la  mission  est 
effectuee.  Cette  cartographie  peut  rester  symbolique  (topologique)  avec  de  grandes 
imprecisions  geometriques. 

Le  second  type  d’activite  des  operateurs  conceme  le  suivi  de  la  mission,  il  s’effectue 
en  ligne  pendant  la  phase  operationnelle,  soit: 

-  La  verification  du  bon  enchainement  des  actions  par  rapport  au  plan  prevu.  Ce  qui 
comprend  les  aspects  topologiques  (positionnement  relatif  du  robot  dans  son 
environnement)  ainsi  que  les  retours  fonctionnels  (etat  interne  du  robot  et  de  ses  sous 
systemes,  etat  de  realisation  des  taches  specifies  ...). 

-  L’ intervention  corrective  de  l’operateur  en  temps  reel  en  cas  de  besoin  (assistance 
au  robot  en  difficult^,  reaction  h.  un  evenement  imprevu). 

-  La  foumiture  au  robot  d’informations  complementaires  permettant  d’enrichir  son 
module  interne  (assistance  au  processus  de  reconnaissance  des  obstacles  ou  amers, 
ajout  dans  la  base  de  donnees  du  robot  d’un  obstacle  non  detecte  par  les  systemes 
embarques,  recalage  assiste  du  positionnement  absolu  dans  l’environnement, 
ajout/modification  d’une  information  de  contexte  ...). 

-  La  confirmation  des  interpretation  et  la  validation  des  intentions  du  robot  comme 
par  exemple:  la  levee  d’ambigui’tes  entre  une  menace  reelle  et  un  leurre,  la  validation 
du  declenchement  d’un  tir  ... 

L’interaction  homme-robot  peut  etre  supportee  par  exemple  par  la  meme  representation 
symbolique  cartographique  sur  laquelle  vont  se  superposer  a  l’avancement  les  informations 
de  compte  rendu  provenant  du  robot  Cependant  des  suppon  differents  et  representations 
complementaires  sont  necessaires  lorsque  1’operateur  doit  intervenir  dans  Ie  processus,  la 
principalc  difficult^  etant  alors  d’obtenir  une  bonne  comprehension  de  la  situation  par 
1’operateur. 

En  effet  lors  d’un  deroulement  nominal  d’une  mission,  le  role  de  l’operateur  se  limite 
a  celui  d’un  superviseur,  se  contentant  de  verifier  que  les  comptes  rendus  sont  coherents  avec 
le  plan  de  mission.  Afin  de  suivre  ce  deroulement  l’operateur  n’a  pas  formellement  besoin 
de  connaitre  de  fa?on  intime  les  details  du  cheminement  logique  du  processus  ddcisionnel  du 
robot,  il  n’a  pas  non  plus  besoin  de  connaitre  la  fag  on  dont  le  robot  se  represente  i’univers 
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qu’ii  rencontre. 

Par  contre  en  cas  d’imprevu  (par  example  une  menace  ennemie  non  modelisee,  ou  bien 
un  obstacle  que  le  robot  declare  infranchissable  et  incontoumable  ...),  le  robot  peut  etre 
bloque  dans  sa  progression  et  demander  l’aide  de  l’operateur.  De  plus  la  notion  de  I’urgence 
de  la  reaction  sera  souvent  associee  (imminence  de  T  agression  de  la  menace,  perte  de  temps 
dans  l’execution  de  la  mission). 

Dans  ce  cas  1’operateur  doit  etre  capable  d’effectuer  un  diagnostic  complet,  juste  et 
rapide  de  la  situation  ce  qui  est  d’autant  plus  delicat  que  sa  vigilance  a  ete  emoussee  lors  des 
phases  se  deroulant  sans  que  son  intervention  dans  le  processus  soit  necessaire.  II  doit 
comprendre  par  exemple  "pourquoi  le  robot  s’est  arrete",  ce  qui  necessite  une  interpretation 
poussee  des  signaux  disponibles.  L’activite  de  1’operateur  ne  peut  plus  se  resumer  a  une 
comparaison  entre  la  situation  escomptee  et  la  situation  presente,  il  doit  prendre  une  initiative 
et  rechercher  activement  des  "explications"  complementaires  lui  permettant  d’apprehender 
l’ensemble  du  contexte. 

2.2  PILOTAGE 

A  ce  niveau  la  tache  a  realiser  par  le  robot  est  decrite  sous  la  forme  de  tmjecioires 
geometriques  reelles  (droites,  cercles ...),  accompagnees  d’ordres  de  configuration  cy  teme 
(valeur  de  gains,  selection  de  modes  de  commande  ...).  Les  donnees  sont  toujours  de  nature 
symboliques  mais  elles  doivent  etre  quantinees  (une  representation  topologique  ne  sufi'r  plus). 

Le  contenu  des  informations  echangees  a  ce  niveau  a  pour  objectif  de  ppdeifia 
l’enchainement  des  instruction  d'un  "lan gage”  ope'rationnel.  Les  instructions  d’un  red  lanrvge 
sont  du  type  suivant: 

-  instruction  fondamentales:  elles  decrivent  le  componement  de  base  du  robot  en 
modes  manuels  ou  automatiques  (ex:  mode  manuel  assiste,  droite  auto  sur  100  metres, 
arret  d’urgence  ...). 

-  instructions  harmoniques:  elles  ne  modifient  pas  le  componement  de  base  mais 
permettent  d’agir  sur  celui-ci  en  agissant  sur  les  parametres  ajustables  (ex:  vitesse 
d’avance,  activation/desactivation  d’un  procede  ...). 

-  instruction  de  gestion  des  evenements:  elles  definissent  d’une  part  les  types 
d’evenements  a  surveiller  de  fapon  asynchrone  (collisions,  combinaison  de  signaux 
capteurs,  horloge  ...)  et  d’aucre  part  ia  re'action  a  adopter  dans  le  cas  ou  I’evenemem 
est  active. 

-  instructions  de  configuration:  instructions  particulieres  orientees  systeme. 

Les  echanges  peuvent  etre  effectues  directement  en  exploitant  la  fomre  brute  de 
description  des  instructions  du  1  engage  (ex:  SET_SPEED  (valeur),  MOVE ...).  Cependam  cette 
formulation  est  peu  expiicite,  directement  issue  du  mdnde  informatique  done  peu  adaptee  aux 
operateurs  qui  ne  sont  pas  des  programmeurs;  de  plus  elle  exige  des  effons  de  memorisation 
et  de  controle  importants. 

L’objectif  de  conception  de  l’interface  operateur  est  de  traduire  1’expression 
"robocentrique"  de  la  tache  (qui  est  represemee  par  ia  sequence  d’ instructions  du  langage)  en 
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une  representation  plus  familiere  pour  1’operateur  de  terrain. 

D’autre  part  le  systeme  assurant  l’interface  doit  etre  capable  de  simplifier  la  tache  de 
I’operateur  en  limitant  le  nombre  d’informations  dchangdes  (definition  de  parametres  par 
defaut,  gestion  de  la  coherence  des  instructions  et  des  sequences  ...). 

2.3  GUIDAGE  ....  ' 

Ce  niveau  de  commande  re^oit  en  entree  la  specification  d’un  emplacement,  ii  assure 
en  sortie  la  generation  du  mouvement  et  son  controle  asservi  en  prenant  en  compte  les 
limitations  physiques  que  presente  l’engin  (cinematique,  dynamique,  saturations ...). 

L’operateur  a  ce  niveau  intervient  principalement  dc  fa?on  "analogique",  e’est  k  dire 
en  specifiant  de  faqon  continue  la  valeur  d’un  ou  de  plusieurs  vecteurs  de  commande 
(telepilotage  de  la  vitesse  et  de  la  direction  par  exemple).  L’inteiface  physique  est  un  manche 
de  commande  qui  doit  s’adapter  aux  caracteristiques  physiologiques  et,  kinesthesiques  de 
1’homme. 


3.  CARACTERISATION  DU  SYSTEME  HOMME/MACHINE 


le  couplage  homme-machine  met  en  oeuvre  des  organes  physiques  qui  adressent  les 
sens  et  les  moyens  d’action  de  l’homme.  Ces  elements  "d’entrees-sorties"  ont  des 
caracteristiques  particulieres  et  des  limitations  qui  vont  guider  la  conception  de  l’interface. 
Comme  le  montre  la  figure  1,  d’apres  A.  Bejczy,  la  premiere  caracteristique  est  la  grande 
dissymetrie  qui  conduit  a  une  saturation  a  cause  des  limitations  en  nombre  et  en  performances 
des  canaux  de  sortie  (envoi  des  informations  depuis  l’homme  vers  le  systeme  commande)  par 
rapport  aux  canaux  d’entree. 


HUMAN 

OPERATOR 


Figure  1:  Canaux  d’entrde-sortie  de  l’homine  (Bejczy) 
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En  effet  en  "entree": 

;  -  La  vue  est  le  sens  majeur  de  perception.  II  est  capable  d’  analyser  en  temps  reel  une 

grande  quantite  d’infonnation  brute  (plusieurs  megabits  par  seconde)  et  de  la  traiter  sur 
plusieurs  niveaux  semantiques  (symboliques  et  analogiques).  Cependant  Thomme  n’est  pas 
capable  de  suivre  plusieurs  processus  simultanement,  de  plus  les  notions  de  fatigue  visuelle 
et  mentale  sont  a  prendre  en  compte. 

Les  images  (directe  video,  synthetique  ou  symbolique)  doivent  etre  de  bonne  qualite 
et  organisees  de  fagon  hierarchique  selon  leur  importance  dans  l’execution  d’une  tache  et 
selon  la  nature  de  la  tache  realisee.  La  saturation  du  canal  visuel  degrade  les  performances 
du  systeme  car  ce  canal  sollicite  beaucoup  le  systeme  nerveux  central. 

-  L’ouie  est  un  canal  moins  performant  en  capacite  par  contre  le  traitement  de 
Tinformation  peut  etre  parallelise.  L’operateur  peut  effectuer  une  tache  principale  en  recevant 
un  retour  sonore  simultane  qui  ne  le  perturbe  pas.  II  peut  etre  utilise  de  fagon  analogique 
(son)  ou  symbolique  (parole).  Ce  canal  est  exploite  de  fagon  tres  importante  pour  la 
communication  entre  operateurs. 

L’ouie  est  utilisee  surtout  pour  la  transmission  d’ informations  de  surveillance 
(detection  de  bruits  anormaux  sur  l’engin)  et  d’alarmes.  C’est  un  canal  efficace  permettant 
de  foumir  des  informations  complementaires  udles  au  processus  de  perception  multisensoriel 
de  Thomme. 

-  Les  sens  kinesthesique  (retour  d’effort)  et  le  toucher  sont  des  sens  analogiques  qui 
sont  utilises  avec  une  classe  particuliere  d’organes  de  commande  actifs  capables  de  restituer 
des  informations  tactiles  ou  d’effort  directement  dans  la  main  de  1’operateur.  Cette  approche 
permet  d’ameliorer  la  transparence  du  couplage  homme-machine  en  exploitam  les 
automatismes  humains  disponibles  au  niveau  de  Tare  reflexe.  Ce  qui  decharge  le  systeme 
nerveux  central.  La  bande  passante  est  assez  importante  (jusqu’a  100  hertz  pour  le  toucher). 

Les  organes  de  "sortie"  sont: 

-  Les  doigts  qui  permettent  d’atteindre  quelques  dizaines  de  bits  par  seconde  par 
frappe  sur  clavier.  La  selection  d’ordres  sur  des  menus  ne  depasse  pas  quelques  bits  par 
seconde  mais  permet  par  contre  d’acceder  a  des  informations  de  contenu  semantique  eleve. 

-  Les  commandes  au  pied  peuvent  etre  udlisees  en  symbolique  ou  analogique 
(accelerateur)  mais  de  fagon  assez  limitee. 

-  La  commande  vocale  permet  de  transmettre  des  ordres  de  haut  niveau  mais  presente 
encore  des  performances  trop  faibles  pour  etre  operationnelle  (taux  de  reconnaissance  et 
limitations  du  vocabulaire). 

-  La  main  et  le  bras  peuvent  imprimer  des  mouvements  de  plusieurs  hertz  de  bande 
passante  analogique  ce  qui  est  tres  utile  dans  le  cas  de  commande  en  teleoperation. 

-  On  peut  citer  encore  des  organes  de  sortie  inhabituels  comme  le  suivi  des 
mouvements  de  l’oeil  et  de  la  tete  qui  peuvent  avantageusement  simplifier  les  problemes  de 
pointage. 
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4.  EXEMPLE  DE  CONCEPTION:  LE  ROBOT  AMR 

Le  projet  AMR  (advanced  mobile  robots),  mene  'dans  le  cadre  du  programme 
EUREKA  a  pour  objectif  de  developper  les  technologies  de  la  robotique  mobile  pour  les 
applications  dans  le  domaine  de  la  securite  civile. 

Le  systeme  est  compose  de  deux  robots,  le  premier  evolue  en  exterieur  et  transporte 
le  second  qui  evolue  a  l’intdrieur  des  batiments.  Us  sont  commandds  a  partir  d’un  poste 
mobile  communiquant  par  radio  avec  les  engins. 

L’etude  des  missions  a  mis  en  evidence  une  grande  destructuration  de  l’environnement 
qui  a  conduit  a  la  conception  suivante  de  la  commander 

-  Le  mode  nominal  de  controle  est  le  mode  teldopere,  I’homme  doit  conserver  la 
responsabilite  directe  des  operation  (necessite  de  vigilance).  L’operateur  dispose  neanmoins 
de  nombreuses  assistances  foumies  par  le  systeme. 

-  Le  fonctionnement  automatique  est  envisage  pour  repondre  aux  situations  frequentes 
dans  lesquelles  le  canal  de  communication  radio  est  degrade. 

La  nature  des  activites  de  commande  necessite  la  presence  de  deux  operateurs.  Le 
premier  se  concentre  sur  les  taches  de  teleoperation  de  l’engin  qui  sont  realisees  soit  en 
telecommande  directe  (guidage)  soit  en  teleoperation  symbolique  (pilotage)  par  la 
specification  de  primitives  de  deplacement.  Le  second  operateur  assiste  le  premier  en  prenant 
<i  sa  charge  les  taches  de  gestionUe  la  mission.  En  effet  les  operations  de  teleoperation 
necessitent  une  concentration  telle  que  les  taches  de  plus  haut  niveau  ne  peuvent  etre  prises 
en  compte  par  le  pilote  seul. 

L’interface  homme-machine,  presente  sur  la  figure  2  est  organise  de  la  fag  on  suivante: 

L’operateur  de  teleoperation  dispose  de  leviers  de  commande  directe  analogique  pom- 
la  commando  du  vehicule  et  des  systemes  embarques  (cameras,  bras  manipulateur).  Devant 
lui  un  grand  ecran  monitem  presente  plusiems  fenetres  ou  sont  affichees  les  images  suivantes: 

-  une  representation  synthetique  3D  de  I’environnement  issue  de  la  base  de  donnees 
locale  domiant  une  vision  complementaire  2t  celle  des  cameras. 

-  plusieurs  representations  synthetiques  extemes  du  robot  pennettant  de  le  situer  dans 
son  environnement  (vues  2D  planes  de  dessus). 

-  diverses  vue  de  television  directe. 

L’operateur  de  controle  de  mission  travailie  devant  un  ecran  graphique  qui  represente 
une  carte  topologique  de  1’environnement.  Cette  representation  est  tracee  par  l’operatem 
pendant  la  phase  de  preparation  de  la  mission  (en  effet  on  ne  dispose  pas  systematiquement 
pom  les  missions  envisages  de  cartographies  precises  de  la  zone  d’intervention),  le  trace 
topologique  permet  de  consumer  un  canevas  general  de  la  mission. 

L’operatem  superpose  sur  cette  representation  un  trace  symbolique  en  precisant  les 
taches  a  rdaliser  a  l’avancement  ("road  book"). 
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TELEOPERATION  MISSION  MANAGEMENT 


Figure  2:  organisation  du  poste  de  commande  du  projet  AMR 


Lors  des  phases  operationnelles  le  robot  fonctionne  soit  en  mode  teleopere  soit  eu 
mode  automadque  en  exploitant  directement  la  descripuon  obtenue  par  le  "road  book"  (niveau 
navigation). 

Lors  du  mouvement  la  representation  topologique  est  affinee  et  renseignee  par  les 
dimensions  reelles  des  objets  tclles  qu’elles  sont  perques  par  le  robot,  ce  qui  permet  de 
construire  le  modele  local  de  l’environnement  exploite  par  les  operateurs  (representations 
svnthetiques  2D  et  3D).  Cette  construction  est  effectuee  de  faqon  interactive  avec  les 
operateurs  qui  ne  retiennent  que  les  informations  les  plus  pertinentes  pour  la  mission, 

Cette  interface  met  en  oeuvre  des  dispositifs  physiques  relativement  standard 
(stations  de  travail  graphiques)  mais  dont  l’utilisation  illustre  quelques  notions  importantes 
evoquees  plus  haut: 

-  les  fonctions  de  navigation,  de  pilotage  et  de  guidage  sont  presentes  et  reparties  entre 
les  deux  operateurs. 

-  Les  images  representees  sont  directement  dependantcs  du  type  d'activite  exercee  (2D 
3D  ou  road  book)  par  les  operateurs. 

-  L’expioitation  de  I’imagerie  synthetique  permet  de  ne  concentrcr  1’attention  que  sur 
les  informations  pertinentes,  et  de  completer  les  retours  video. 

-  La  construction  interactive  du  modele  local  qui  est  utilise  pour  produire  les  images 
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synthetiques  permet  de  construire  un  modele  d’environnement  "oriente  operateur".  La  vision 
"robocentxique"  est  interpretee  et  transformee  en  une  representation  plus  directeraent  explicite 
pour  les  operateurs  humains. 

5.  CONCLUSION 

Les  systemes  robotises  militaires  doivent  rester  en  relation  plus  ou  moins  permanente 
avec  un  poste  de  commande  afin  d’assurer  leur  suivi  par  un  ou  plusieurs  operateurs. 

Le  haut  degre  d’automaticite  qu’il  est  souhaitable  d’atteindre  a  terme  -  afin  de 
diminuer  l’investissement  et  done  l’exposition  des  operateurs  humains  sur  le  champ  de  bataille 
•  ne  signifie  pas  que  la  problematique  de  i’interface  homme-machine  s’estompe.  Les  ordres 
de  haut  niveau  doivent  etre  produits  de  fapon  sure. 

Les  techniques  de  base  de  conception  des  interfaces  physiques  sont  disponibles, 
cependant  elles  sont  encombrantes  (dimensions  importantes  des  postes  de  terrain  congus  h 
partir  de  consoles  et  d’ecrans  standards),  ce  qui  justifie  une  recherche  de  compacite 
notamment  en  suivant  les  efforts  developpes  en  aeronautique  (poste  virtuel  intdgre  dans  le 
casque). 

Les  techniques  d’interfagage  au  niveau  du  traitement  et  de  la  presentation  de 
I’ information  sont  par  contre  actuellement  mal  maitrisees  et  reposent  sur  des  approches 
empuiques.  n  est  souhaitable  que  cette  connaissance  soit  approfondie  au  niveau  th6orique 
ainsi  qu’au  moyen  d’experimentations  en  vraie  grandeur. 
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L'apprentissage  du  pilotage  d'un  vdhicule  fait  appel  k  l’acquisition  d'une  connaissance  implicite  des 
particularitds  statiques  et  dynamiques  du  vdhicule  lui-meme,  autour  de  chacun  de  ses  points  de 
fonctionnement,  mis  en  regard  de  l'enviionnement  immddiat  dans  lequel  il  dvolue. 

Cet  apprentissage  dtant  acquis,  l'opdrateur  exploite  la  richesse  de  ses  capteurs  physiologiques  pour 
apprecier  les  reactions  fines  de  son  vdhicule,  aux  ordres  instantands  qu'il  lui  donne,  mais  dgalement 
aux  sollicitations  en  provenance  de  son  environnement. 

Ces  informations,  en  retour  du  vdhicule  et  de  son  environnement  immddiat  proviennent  au  pilote : 

-  par  ses  capteur  auditifs :  rdgime  moteur,  puissance  foumie,  roulage  du  vdhicule, 

-  par  ses  capteurs  inertiels  :  mouvements  instantands  du  vdhicule,  ses  vitesses  angulaires  de 
tangage,  roulis  et  lacet,  ses  accdldrations  lindaires  (verticale,  axiale  et  latdrale)  et  son  dquilibre 
instantand  (verticale  locale), 

-  par  ses  capteurs  visuels  :  environnement  futur,  immediat,  vitesse  du  vdhicule  (vision 
peripherique),  position  du  vdhicule  relativement  a  la  trajectoire  a  realiser  (eireur  laterale,  eireur  de 
cap  instantand). 

Un  des  interets  de  la  tdld-opdration,  relativement  k  l'autonomie  complete,  d'un  mobile  devant  dvoluer 
dans  un  environnement  non  structure,  mal  connu  et  potentiellement  hostile,  est  de  chercher  k  exploiter 
les  capacites  d'un  operateur  a  apprecier  instantanement  une  situation  globale  complexe. 

Cet  objectif  se  heurte  a  la  triple  necessite  de  disposer : 

1.  -  de  capteurs,  permettant  d'apprehender:  l'environnemcnt  du  vdhicule,  son  comportement 
instantand  et  son  dtat  interne, 

2.  •  d'une  ligne  de  transnussion  liable,  a  large  bande,  entre  le  mobile  et  le  poste  de  tdld-opdration. 

3.  -  de  moyens  physiques  adequats  permettant  de  communiquer  ces  informations  a  1'opdrateur  dans 

le  but  d'exploiter  les  automatismes  qu'il  aurait  acquis  par  apprentissage. 
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Le  but  de  cet  expose  va  etre  de  chercher  k  identifier  les  traitements  ou  modifications  locales  a 
envisager  au  niveau  du  vdhicule  pour  en  am&iorer  le  compoitement  nature!,  dans  le  but  de  minimiser 
les  besoins : 

-  d'interfaces  riches  au  niveau  de  l’operateur, 

-  d'apprentissage  de  la  complexity  physique  du  systfcme  et  des  moyens  d’interfa^age, 

-  en  disponibility  de  1'operateur. 

On  va  ainsi  faire  apparaitre  deux  niveaux  de  traitements  qu'on  chercherait  a  rdaliser  localement  sur  le 
vehicule  et  permettant  de  trailer  respectivement,  des  anomalies  lides  a  la  technologie  interne  du 
vehicule,  et  des  effets  des  interactions  directes  entre  le  vehicule  et  son  environnemen:  immediat 
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2.  “  MODIFICATION  DES  CARACTERISTIQUES  ISQ-COMM  ANDES 
"INTERNES"  DU  VEHICULE 

Dans  tout  vthicule  on  retrouve,  rtalistes  au  moyen  de  technologies  plus  ou  moins  complexes,  les 
trois  fonctions  nominales :  propulsion,  freinage  et  direction. 

Afin  de  simplifier  la  commande  du  vthicule  on  cherchera  a  faire  disparaitre  la  rtalitt  technologique  du 
systeme  sous  deux  types  de  caractdristiques  iso-commandes  d'avancement  et  de  direction,  fonctions 
de  la  vitesse  du  vthicule,  mais  respectant  la  forme  des  limitations  internes,  extemes  ou 
ergonomiques. 

2.1.  -  COMMANDE  D'AVANCEMENT 

Pratiquement,  tous  les  vthicules  i  mobility  moyennement  tlevte  disposent  d’une  chaine  de 
propulsion  constitute  d’un  moteur  thermique  suivi  d'un  systtme  de  transmission  de  la  puissance, 
commandable  de  fa?on  continue  ou  discontinue,  l'ensemble  donnant  acces  a  un  domaine 
Force-Vitesse,  bomt  : 

a.  -  Au  sommet  du  1CT  quadrant  par  l'effort  propulsif  maximal  disponible. 

b.  -  Par  une  courbe  de  forme  iso-  puissance  obtenue  en  combinant  l'enveloppe  des  caracteristiques 

du  moteur  et  les  rtglages  continus  ou  discontinus  de  la  transmission. 

c.  -  A  droite  par  la  vitesse  d’avancement  maximale  du  vehicule. 

A IOTA  : 

Sur  les  vihicules  d  transmission  discontinue  et  convertisseur  de  couple  non  commandable,  la 
presence  d'une  trainee  de  convertisseur  impose  d'utiliser  la  commande  de  freinage  pour  comroler  le 
triangle  de  foibles  vitesses  et  efforts  propulsif s  (cf.  figure  2.1.1 
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Figure  2-1-1-  Domaine  accessible  par  la  commande  du  systeme  propulsif 

Une  premiere  partie  du  quadrant  4,  correspondant  aux  forces  ddcdldratrices  est  foumie  par  le  frein 
moteur  qui  peut  avoir  une  allure  fortement  hachee,  en  particulier  sur  un  vehicule  a  rapports  de 
transmission  discontinus. 

Au  dela,  les  forces  moyennes  et  elevees  de  ralentissement  ou  de  freinage  sont  foumies  par  des  freins 
mecaniques,  combines,  sur  les  vehicules  lourds  h  des  ralentisseurs  electriques  ou  hydrauiiques 
dimensionnes  pour  evacuer  des  puissances  tliermiques  permanentes  elevees. 

L'effon  maximal  de  freinage  di^pomble  est  en  general  impose  par  les  caracteristiqucs  d'adherence 
maximales  sur  sol  dur. 

A  I'interieur  du  domaine  F-V  accessible,  les  caracteristiques  iso-commandes  "naturellcs"  du  systeme, 
dependent  de  la  technologic  et  des  lois  de  commande  locales  miscs  en  place  sur  le  moteur  et  sur  la 
transmission  pom*  minimiser  les  risques  d'dtouffement  et  d'emballement  du  moteur  ou  simplement 
pour  obtenir  un  point  de  fonctionnement  stable  de  la  transmission  (dans  le  cas  de  transmissions 
hydrostatiques). 
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Figure  2-1-2-  Caracteristiques  iso-commandes  naturelles  d’un  vehicule 

Sur  la  figure  2.1.2.  on  a  reproduit  un  ensemble  de  caracteristiques  iso-commandes  d’acceterateur 
reprdsentatif  d'un  vehicule  a  rapports  de  boite  discontinus,  et  muni  d’un  convertisseur  de  couple.  La 
force  d’avancement  agissant  directement  sur  l'acceleration  du  v6hicule,  la  pente  locale  de  ces 
caracteristiques  est  macroscopiquement  le  reflet  de  la  constante  de  temps  de  mise  en  vitesse  du 
vehicule  qui  vane  de  0,15  s  (rapports  courts,  force  d'avancement  faible)  4  6s  (vitesses  dlevees,  et 
convertisseur  de  couple  en  action). 

Dans  le  but  de  simplifier  la  commande  mise  a  la  disposition  de  1'operateur,  et  de  minimiser  les 
besoins  en  contre-rdactions  fines  4  lui  presenter,  on  detruit,  dans  un  premier  temps  les 
caracteristiques  initiales  du  systfcme  en  supprimant  la  contre-rdaction  de  vitesse  du  moteur  et  en 
gcnerant  artiflciellement  des  courbes  iso-commandes  ayant  la  forme  gdnerale  represcntee  sur  la  figure 
2.1.3. 

Sur  cette  figure,  la  force  d'avancement  maximale  correspond  4  unc  acceleration  du  vehicule  de 
I’ordre  de  7  m/s2 .  Les  iso-commandes  d'avancement,  appliqudes  sur  le  vehicule  font  done  apparaitre 
une  constante  de  temps  de  mise  en  vitesse,  de  1'ordre  de  2  secondes,  approximativement  constante 
dans  la  plus  grande  panie  du  domaine  accessible.  On  redendra  que  cet  ordre  de  grandeur,  faible  pour 

tin  vehicule  do  competition.  ext  hien  adapts  pour  une  utilisation  en  teifcornmantte 
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Dans  le  systeme  experimental  realise,  ia  force  propulsive,  calcuiee  a  partir  de  la  consigne  de 
l'operateur  et  de  la  vitesse  d'avancement  du  vehicule,  est  obtenue  par  choix  convenable  de  la 
commande  d'injection,  prenant  en  compte  les  caracteristiques  internes  du  moteur,  la  vitesse 

•i  . ' 

instantanee  du  moteur,  et  l'dtat  de  la  bolte  de  vitesse. 

Vitesse(Km/h)  ; 


Commande 

Figure  2-1-4  -  Augmentation  de  la  sensibilite  de  la  commande  aux  vitesses  faibles 

Pour  les  iso-commandes  correspondant  aux  faibles  forces  et  vitesses  d'avancement,  l’automatisme  ira 
solliciter  la  commande  de  freinage,  ce  qui  permettra  de  masquer  a  l'operateur  l'anomalie  locale  due  a 
la  trainee  du  convenisseur  de  couple. 

Autour  des  vitesses  faibles  (0-10  km/h),  ou  une  plus  grande  sensibilite  de  la  commande  peut  etre 
necessaire  pour  niieux  maitriser  les  manceuvres  du  vehicule,  on  augmentera  la  sensibilite  locale  en 
introduisant  une  correspondance  du  type  bilineaire  ou  cubique  entre  le  manipulateur  et  la  consigne 
d'avancement  du  vdhicule.  Une  augmentation  dans  un  rapport  de  3  H  5  du  gradient  local  autour  du 
zero  est  bien  perdue  ergonoraiquement  (cf,  figure  2.1.4.). 

Au  passage  par  zero  de  la  force  axiale,  la  pente  des  iso-commandes  d’avancement  (==  0,5  m/s2/m/s) 
est  conservee  dans  un  domain?  correspondant  approximativement  au  firein  moteur  moyen.  Les  iso- 
commandes  d'avancement  sont  done  bomees,  en  effets  negatifs,  par  une  caracteristique  iso- 
commande  nulle. 

Les  iso-commandes  de  freinage  sont  obtenues  par  translation  lineaire  de  1'iso-commande  nulle 
(efiigure  11.5.). 
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2.2.  -  COMMANDE  DE  DIRECTION 

La  comtrinde  de  direction  aes  vehicules  £  roues  directrices,  et  d  une  ptrtie  des  vehicules  a  direction 
par  "skid-steering"  fait  natureileraent  correspondre  un  rayon  de  virage  a  une  consigne. 

Pour  les  vehicuies  chenilles  dans  lesquels  la  coramande  differentielle  des  chenilles  est  realises  par  le 
pilotage  de  la  cylindree  d’un  groupe  hydrostatique ,  le  gradient  entre  la  consigne  de  virage  et  le  rayon 
realise  varie  en  fonction  du  rapport  de  reduction  enclenche,  mais  pour  un  rapport  donne  reste  en 
premiere  approximation  constant 

Aux  faibles  vitesses  d'avancement,  le  domaine  des  virages  accessibles  par  la  plupart  des  vehicules  est 
done  borne  par  le  rayon  minimum  realisable  technologiquement. 

Des  que  ia  vitesse  d ’avanceoicnt  augmente,  les  possibiiites  reeiles  de  virage  sont  limitees  par  des 
considerations  ergonomiques  entre  0,6  et  1  rd/s. 


V  itessc  d’avancement 
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Lorsque  la  vitesse  d'avancement  augmente  encore,  les  possibilitds  de  virage  sont  limitees  par  les 
conditions  d'adhdrence  entre  le  vehicule  et  le  sol  qui  boment  l'accdldration  centripete  maximale  aux 
alentours  de  9  m/s2  sur  un  vehicule  k  roues  directrices  et  vers  6  m/s2  sur  un  vdhicule  chenilld. 

L’exploitation  de  ces  considerations  k  la  tdldcommande  d'un  vdhicule  debouche  sur  des 
caracteristiques  iso-commandes  de  virage  correspondant  k : 

-  un  rayon  de  virage  constant  aux  faibles  vitesses, 

-  une  vitesse  angulaire  constante  aux  vitesses  moyennes, 

-  et  une  acceleration  laterale  constante  aux  vitesses  elevdes. 

Ce  type  de  caracteristique  permet  d'exploiter  le  debattement  utile  du  manipulateur  de  commande  dans 
tout  le  domaine  des  vitesses  d'avancement  accessibles. 
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3. -ACTIONS  SUR  LE  COMPORTEMENT  PU  VEHICULE.  INFLUENCE 
PAR  SON  ENVIRONNEMENT  IMMED1AT 

Un  pilote,  par  action  sur  ses  commandes  d'avancement  et  de  direction,  agit  sur  son  vdhicule,  pour 
controler  sa  position  et  sa  vitesse  futures,  sur  un  horizon  de  quelques  secondes. 

Derriere  cette  strategie  k  moyen  terme,  Ie  pilote-  doit  reagir  k  des  reactions  instantanties  de  son 
vehicule,  consequences  d'une  mauvaise  combinaison  de  ses  caracttiristiques  intrinsdques,  des 
consignes  donnees  par  le  pilote,  et  de  l'environnement  immddiat. 

Les  reactions  instantanees  inddsirables  du  vdhicule  sont  constitutes  par  la  perte  du  controle  de  son 
lacet,  et  de  son  roulis.  On  idmettra  que  la  perte  de  controle  du  tangage  du  vehicule  est  du  ressort  de  la 
strategie  a  moyen  terme. 

3.1.  -  STAB1UTE  DU  LACET  D'UN  VEHICULE 

La  stabilite  du  lacet  d'un  vehicule  telecommande  est  une  caracteristique  qu'on  devra  rechercher,  dans 
la  mesure  oil  1'appreciadon  de  la  qualite  du  terrain,  aussi  bien  que  les  dosages  respectifs  des  ordres 
de  freinage,  d'acceldradon  et  de  direction  ne  peuvent  etre  qu'approximatifs.  On  attachera  done  une 
grande  importance  dans  l'analyse  de  ces  phenomenes  et  dans  la  recherche  des  solutions  qui 
permettraient  aussi  bien  de  forcer  le  vehicule  a  ne  pas  diverger  par  rapport  aux  consignes  qu'il  revolt 
du  pilote  (a  l'interieur  du  domaine  accessible),  ou  d'interdire  au  vehicule  de  realiser  les  consignes  qui 
lui  sont  donnees  lorsque  celles-ci  1'entraineraient  k  1'exterieur  de  son  domaine  instantanement 
accessible  (mise  en  derapage). 

3.2.  -  CAS  PARTICULAR  DES  VEHICULES  CHENILLES 

Les  vehicules  chenillds  ou  k  roues  non  directives  constituent  une  famille  interessante  pat  ses  capacites 
d'evolution  en  terrain  difficile. 

Ces  vehicules  se  particularisent  par  le  fait  que  le  synchronisme  des  roues  ou  galets  d’un  demi-train  est 
assure  mecaniquement  et  que  les  settles  commandes  accessibles  sont  respectivement  le  mode  commun 
et  le  mode  difftirentiel. 

Les  vehicules  a  commande  differentielle  de  la  vitesse  des  roues  ou  chenilles  presentent  naturellement 
la  particularity  d'etre  instables  dans  une  paitie  du  domaine  des  virages  potentiellement  accessibles  sur 
un  terrain  de  coefficient  de  glissement  chenilles-sol  donne. 
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Ce  domaine  est  reprdsent6  schdmadquement,  sur  la  figure  3.2.1  dans  un  systfcme  de  coordonndes  lie 
a  la  trajectoire. 

Dans  ce  domaine  de  virages  instables,  le  vdhicule  part  naturellement  en  survirage  et  referme 
progressivement  sa  trajectoire.  Cette  instability  serait  att6nu6e  par  une  alteration  violente,  mais  la 
puissance  disponible  est  en  gdndral  h  peine  suffisante  pour  entretenir  le  virage  h  vitesse  dlevde. 
L 'amelioration  de  la  stability  du  virage  en  prdsence  d’une  acceleration  est  done  iliusoire. 


Figure  3*2-1-  Domaine  de  stability  naturelle  d'un  vdhicule  cbeniliy 


Les  courbes  reprysentees  sur  la  figure  3.2.2.  donnent  l’allure  des  caractdristiques  stadques  qui  relient 
la  vitesse  difidrentielle  des  chenilles  et  la  vitesse  angulairc  du  vehicule  et  de  sa  trajectoire,  sur  laquelle 
le  domaine  instable  correspond  h  la  partie  4  pente  odgative. 
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NOT  A  : 

Le  schema  de  la  figure  3.2.1  laisserait  supposer  que  tout  le  domaine  grise  est  theoriquement 
accessible,  sous  reserve  d'une  commande  de  direction  adiquate.  Ceci  n'est  que  partiellement  exact. 
En  effet,  on  retrouve  bien,  sur  ce  type  de  vehicule  le  meme  genre  de  phenom&ne  que  sur  les  vehicules 
a  roues  directrices  et  qui  concerne  Impossibility  tMorique  et  physique  d'exploiter  totalement  iu 
possibility  maximale  d'effort  tangentiel  v£hicule~sol  pour  combiner  me  d&ciiyration  le  long  de  la 
trajectoire  et  une  force  centripdte  importante,  en  presence  d'un  transfert  de  charge  axial  dud  la  hametr 
du  cemre  de  gravity. 

Sur  ia  figure  3.2.3.on  a  represente  les  modifications  des  caract6risdques  obtenucs  par  l'utilisatior. 
d'une  loi  de  commande  qui  exploite  la  mesure  des  efforts  lateraux  sur  les  galets  extremes.  On  notcra, 
en  particulier,  que  ce  type  de  loi  de  commande  permet  de  retrouver  une  relation  biunivoque  entre  la 
commande  donnee  par  le  pilote,  et  la  vitesse  angulaire  realisee  par  le  vehicule,  et  ce,  en  particulier  er. 
presence  d'une  force  de  deceleration  .On  imaginera  son  interet  sur  un  vdhicule  teiecommande  qui, 
malgre  les  precautions  prises  par  le  pilote,  peut  sc  trouver  surpris  par  une  diminution  brutale  du 
coefficient  d’adherencc  qui  peut  momentanemem  placer  le  vehicule  dans  son  domaine  instable. 
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Figure  3-2*3-  Stabilisation  du  virage  d'un  vehicule  chenille  par  conrre-reaction  d'efforts  latdraux 


En  reality,  c'est  surtout  sur  la  limite  du  domaine  que  la  pene  du  controle  du  vdhicule  risque 
d'apparaitre.  En  effet,  des  que  les  possibility  raaximales  de  virage  sont  atteintes  (limite  d'adherence), 
la  vitesse  angulairc  du  vehicule  continue  d'augmenter,  alors  que  la  vitesse  angulaire  de  la  trajectoire 
est  bloquee  a  sa  valeur  maximale.  Sur  un  vehicule  chenille,  on  montre  dgalement  que  la  mise  en  place 
d’une  tres  forte  force  propulsive  aurait  un  effet  stabilisant,  dans  le  sens  que  cet  effort  a  tendance  a 
faire  diminuer  la  vitesse  angulaire  du  vehicule,  raais  cet  intdret  n'est  toujours  que  thdorique,  dans  la 
mesure  ou  ce  type  de  vdhicule  ne  dispose,  en  gdndral  que  d’une  faible  force  propulsive  h  grande 
vitesse. 

On  reticndra  plutot  que  tout  effort  de  freinagc  applique  sur  les  chenilles,  alors  que  le  vehicule  a  atteint 
sa  limite  d’adherence,  deelenche  instantaneraent  un  depart  en  survitesse  angulaire,  amplifie  par  la 
presence  d’un  transfer!  de  charge  axial. 

Des  etudes  sont  en  cours,  d’une  part  pour  completer  la  connaissance  thdorique  du  componement  d'un 
vehicule  chenille  en  virage  sous  forte  deceleration,  et  d'autre  part  sur  la  recherche  des  moyens 
permcttant  de  limiter  automatiquement  l'incidence  du  vehicule  sur  sa  trajectoire  lorsque  des  ordres 
superieurs  a  ses  possibility  lui  sont  donnes. 
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3.3.  -  STABiUTE  DYNAMIQUE  DE  ROULIS 

Les  vehicules  destines  k  se  ddplacer  en  terrains  varies,  k  des  vitesses  moyennes  (10  k  15  m/s), 
doivent  disposer  d'une  garde  au  sol  importante  qui,  associee  k  une  largeur  limitee  les  rend  a  priori 
sensibles  au  ddvers  dynamique  du  terrain. 

Ces  vdhicules  sont  sensibles  a  des  formes  particulieres  du  terrain  qui  font  apparaitre  des  vitesses 
angulaires  de  roulis  importantes  qui  peuvent  etre  k  l'origine  de  decollements  d'un  demi-train. 

Si  ces  variations  dynamiques  d'efforts  sont  combines  aux  transferts  de  charge  latdraux  apparaissant 
en  virage,  on  augmente  les  risques  de  retoumement  du  vehicule. 

Sur  un  vehicule  pilots,  ce  risque  est  relativement  limits,  d'une  pan  parce  que  le  pilote  voit 
l'ondulation  du  terrain  qui  va  sollicker  sa  suspension,  et  d'autre  pan,  apprecie  les  reactions  de  son 
vehicule,  sollicite  par  le  terrain  et  par  les  consignes  de  direction  qu'il  a  pu  donner  pour  sa  propre 
strategic  de  pilotage  et  pour  aider  le  vehicule  a  absorber  l’anomalie  du  terrain. 

En  teldcommande,  l’appreciation  precise  du  terrain  immediat  est  delicate,  et  les  retours  instantanes  du 
vehicule,  difficiles  et  couteux  a  mettre  en  oeuvre. 


La  stability  dynamique  du  roulis  de  certains  vehicules  telScommandes  devrait  pouvoir  etre 
artificiellement  amelioree  par  l’exploitation  de  capteurs  de  terrain  futur  (a  une  dizaine  de  metres)  qui 
sentient  exploites  pour  aider  le  vehicule  k  negocier  l'obstacle. 
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•  algorithms  A  ir  )  de  pratotyaas-rabats  mdbilas  a  deplacement 
(provisoirement J  discret  sur  aattes  (verins),  at,  quand  cas 
robots  serant  develaooes  3  dimension  "champ  da  bataiile"  et 
eauiaes  de  cameras,  a  1 'identification  d'objectifs  (cnars)  oar 
circuits  neuraux.  Les  resultats  accuis  sont  orssentes  oar 
cette  premiere  contribution. 

(■H)les  des  pays  de  l'OTAN  et  a  leurs  faurnisseurs  travaiilant 
sous  centrat  sn  bonne  et  due  forme.  L 'autcrisation  du  Chef  ds 
la  Section  Recherches  oaur  la  Defense  sst  recuiss  oaur  la 
diffusion  a  d'autres  dastinataires. 


1 ,  Introduction 


La  robotique  militairs  s'interesse  a  trois  tyoes  de  fonctions, 

1 'information ,  le  combat  at  la  lagistique.  Chacune  d 'alias  peut  §tre 
caractsrisee  oar  differsntas  missions  confiees  a  des  systemes  stati- 
ques  ou  mobiles  partisllement  cu  totalement  rabotises.  Ces  fonctions 
et  missions  ant  ete  decrites,  notamment,  dans  (l)  et  resumees  oar  le 
tableau  1.  Les  axes  de  recherche  sausentendent  une  collaboration  pluri- 
disciplinaire,  Nous  avons  souligne  ceux  qui  nous  oreoccupent  en  priori¬ 
ty,  et,  parmi  oux,  1 'identification  d'objectifs  par  traitement  d'images, 
a  partir  de  reseaux  neuronaux  dans  l'infrarouge  thermique,  le  develoo- 
oement  de  prototypes-robots  mobiles  de  laboratoire  en  vue  de  leur  con- 
trQle  a  partir  de  systemes-experts.  Cette  premiere  contribution  presen¬ 
te  ces  deux  aspects. 

Tableau  1.  Fonctions ,  missions  et  axes  de  recherche 


Fonction 

Missions 

Axes  de  rechercne 

Information 

Designation  des 
objectifs,  iden¬ 
tification  , 
svstames  0*32.51*^2 
Renseignement 

Developoement  des  caotaurs 

Traitement  d'imaaes 

Systemes  staticues  autonomes  crien- 
tables  ou  tsle-orientables , 

Systames  mobiles  autonomes 
Intelliaence  artificiells 
Telecommunications  ' 

Traitement  de  la  oc  jle 

Comcat 

Chars  rcoctisss  !  Mecanicue,  dvnamioue, simulations 
Artillene  auto-  {  Actuateurs  hvdrauliaues  ou  autres 
metrics, minacs,  |  Matsriau,  blindage,  caDteurs... 
deminage  1 

logisrique 

Accrovisiennement 
Reoaraticns , 
decontamination 
deblaiements 

Mobilite  des  roocts 

Bras  articules  oolyvaients 
Cinematique  et  dtynamioue 

Automates 

Servocommandes , etc . 

2.  Of velcorements  de  rodeos  -.ociles, 

2.1.  arc hi ten to re 

Les  prototypes  deueloooes ,et  actuel- 
Isment  utilises  an  envirenrement  intsrisur  ou  en  site  extfrieur 
orooro,  dpi vent  oarmettre  1* etude  detaillae  de  leur  cinematicue, 
ca  leur  ccmoartement  dynamicue,  de  laurs  oropriitss  mecaniaues  en 


AC/243-TP/3 


C12.2 


vue  d'une  aaaotation  ootimale  a  des  conditions  tout-terrain.  Ils 
ccivent  egalement  oemettrs  1' etude  tie  diff Brents  types  de  caDteur, 
la  mise  au  ooint  de  cartes  electroniques  d1 interface  vers  des 
ordinateurs,  la  deveioooement  de  systemes-experts  de  contrSle  et 
de  commande.  La  demarche  aue  nous  avons  adoptee  est  an  fait  voisine 
de  cells  qui  oreside  au  develappement  du  pro jet  EurSka  AMR  (2)  ou 
du  projet  PAMIR  (3). 

2.1.1.  Robot  mobile  cartssien  a  2  deorss  de  liberts. 

Le  oremier  prototype  develoooe,  dont  la 
figure  1  ci-dessous  presents  une  vue  eclates,  presente  un  cadre 
portant  de  550  mm  de  cQta,  assurant  la  mobilite  de  1' ensemble,  un 
support  mobile  de  sonde  st  un  panneau  de  commande.  Le  cadre  est  oour- 
vu,  sous  chaque  cfits,  d'un  verin  oneumatioue  sans  tige  double-effet, 
dont  la  curseur  deolace  une  bequills  horizontals  elle-m§me  pourvue 
a  ses  extremites  de  deux  verins  elsvateurs  double-effet  termines  oar 
une  ventouse,  Lorsque  les  ventouses  sent  activees,  alias  maintiennent 
las  beauillas  dans  laur  position  couranta  at  1 'alimentation  sn  air 
comcrime  de  deux  verins  sans  tine  parallsles  provoqua  la  deolacement 
du  cadre t  la  commande  d'un  tal  mouvement,  possiola  suivant  deux 
directions  orthogonalss,  est  done  seouentielle.  Ella  est  resumes  an 
six  etaoes  oar  la  figure  2.  La  terms  "becuills"  est  chcisi  oar  analo- 
gia  au  mode  de  oeolacament  utilise  par  un  unijambista  s 'aidant  de  deux 
beouillss  ou'il  deolace  d'atoord  vers  1'avant  (ataoas  1  a  d)  avant  da 
deolacer  son  oroore  coros  (ici,  la  cadre)  dans  cette  m§me  direction. 
Las  verins  elevataurs  osrmettant  d' autre  part,  si  necessaire,  la  fran- 
cnissement  ds  faiblss  denivsilations.  Le  robot  peut  §tre  eauioe  d'un 
systems  imoose  oar  la  mission  qui  lui  est  confiee,  Dans  ca  cas,  11 
oossada  un  systems  orthogonal  tridimensiennei  da  balayags  osrmettant 
la  oeolacamant  d'une  sonce  sur  la  surface  dslimitse  oar  le  caare. 
Enfin,  le  canreau  de  commanca  suooorte  les  distributsurs  electro-oneu- 
matiaue,  le  cottier  electricue  des  connexions  vers  ces  distributsurs 
et  vers  les  oaotsurs  i.nductifs  fin-de-course  du  systems  de  talayacs, 
vers  un  caoteur  a  ultrasens  monte  au  sommet  ds  os  oanneau  et  destine 
au  balayags  exterocsotif  de  1 'environnemant  du  rocot. 

La  commande  du  robot. est  assures  oar  un  automate  transitiornsl  pro¬ 
grammable  tandis  qua  las  imoulsians  nscessaires  a  1 'activation  de 
est  automats  sont  dalivres  oar  un  PC  395  au  reside  le  systeme-exoert 
decrit  ulterieurement. 

Le  rooct  est  actuellsmsnt  oourvu  d'un  cordon  ontilica!  d 'alimentation 
en  2a  V  at  er  air  comorime.  Son  adaptation  aux  conditions  exteriauras 
imolicusrait  une  notarisation  dp  base  electricue  ou  cnsmcaynanicue , 

Is  rsmolacement  des  verins  oneumatioues  oar  leur  equivalent  amoiifico- 
taur  hydraulique,  enfin  Is  rsmolacement  des  ventouses  oar  des  roues  ou 
oar  des  chenillettes . 

Le  robot  ainsi  develoooe  case  50  daN.  Les  deolacemencs  sont,  oour  une 
sequence  de  5  etaoes,  quantifies  3  20,08  cm  et  29,38  cm  en  moyenne: 
oas  valeurs  liees  2  la  course  des  verins  sont  differences,  en  X  et  Y, 
oour  svitor  la  collision  des  bequilles,  et  mingnt  h  une  precision 
infsrieure  a  5/10. 000. 
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annexe,  la  photo  1  oresente  une  vue  "ouverte"  du  robot  baptise 
ALR1JA  (automate  mobile  pilots  oar  une  reconnaissance  oar  ultrasons 
et  1  algorithms  A*),  tandis  oue  la  ohoto  2  decrit  le  systeme  de 
balayage:  ce  dernier  oermet  de  destiner  le  robot,  adaote,  a  troi*= 
genre  de -minions  deminage  oar  detection  surfacique,  decontamination 
par  asoersian  surfacique,  insoection  logistique  en  dotant  la  sonde 
de  caoteurs  aonroories.  D'autres  types  de  mission  peuuent  aussi  @tre 
imaginees.  L 'archi tecture  mecanioue  est  incontestablement  avantaqeuse 
oour  1  eyitement  d’obstacles  si  on  la  comoare  a  celle  he  vehicules  a 
r°U!*  a  neanmoins  cherche,  en  conservant  le  orincioe  des  beouilles 
olutSt  (au  stade  actuel)  cue  cslui  des  jambes  (legs),  a  develaooer 
une  structure  a  trois  degres  de  liberte. 

2,1,2,  Robot  mobile  a  3  degres  de  liberte. 


8antise  ROMOCAN  (robot  mobile  Dilate  oar  camera  et  circuit  neuronique) 
xl  est  schematiouament  decrit  oar  la  figure  3  ci-dessous. 

San  architecture  est  constituee  de  trois  grandes  oarties:-un  oied 
central  attache  sous  -  un  plateau  intermedia! re,  le  tout  surmon- 
te  d un  cadre.  La  robot  oeut  reooser  sur  deux  grouoes  de  ventouses, 
trois  ventouses  sous  le  aied  central  ou  auatre  ventouses  fixees  au 
bord  exterieur  du  cadre. 
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(_es  mouvements  relatifs  entre  les  differentes  oarties  nrincioales 
sont: 

-  une  rotation  3  de  1' ensemble  nlateau  intermpdiaire  et  cadre 
autour  riu  Died  central  lorsuue  les  ventouses  de  ce  dernier 
sont  active0? ,  ou  inversement,  une  rotation  riu  oied  central 
cuanri  le  rocot  sst  fixe  oar  les  ventouses  exterieures  du 
cadre. 

-  une  translation  T  de  1 ' ensemble  nlateau  intermpdiaire  et  oied 
central  oar  raooort  au  cadre  lorsnue  le  robot  est  nose  rur  les 
"oattes"  exterieures  de  celui-ci,  et  inversement  nuanri  le  robot 
est  aoouyp  sur  le  oied  central. 

Trois  moteurs  electricues  A  courant  continu,  pauiops  de  reducceur, 
assurent  les  mouvements  nrecites  auxauels  s'aioute  la  rotation  de  la 
camera  fixee  au  sommet  du  cadre  oyramidal.  Les  oattes  exterieures  ront 
Piles  des  verins  elevateurs  double-pffet.  Line  rotation  tie  ces  verins 
autour  ri'un  axe  Horizontal,  nernendiculaira  a  I'axe  T  reoris  sur  '.a 
ftqure  3,  oermettra  uiterieurement  d'eturiier  le  comoortement  du  robot 
"ouariruoAde" ,  La  odotd  3  de  1 'annexe  donne  une  vue  d'ensemble  du  robot 
tanriis  nue  la  ihoto  :l  omsente  la  matoreducteur  assurant  la  translation 
du  cadre  via  cre^aillam  et  donne-  t^qalement  une  vue  du  moteur  de  came¬ 
ra  et  d'un  verin  exterieur.  He  robot,  fonctiannant  sur  les  mGmes  orin- 
cioes  nue  ceux  de  1  '-VVfllJA,  nermet  ceoenriant  un  deiolacement  dans  tQUtes 
les  directions,  sur  une  surface  nlane  ou  leoarement  canvexe  ( l?Q  mm  rg 
rayon  ds  courbure)  ou  concave  (S’M  mm),  L 'ensemble  nose  environ  P  duN 
et  oeut  Stre  enveioooe  (camera  incluo)  dans  un  parallel ioioede  recton- 
qle  de  30  x  2m  x  3<1  cm.  II  est  directement  commando,  via  cordon  ombili- 
cal  ‘Hectrioue  er  nneumatinue ,  A  oartir  d'un  PQ  3P*3,  II  est  destine  A 
la  reconnaissance?  o'rcisctifs  -  ou  o' obstacles)  oar  camera. 
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2.2.  Electroniaue  st  informatiaue. 

Les  deux  prototypes  sont  electra- 
oneumatiques  et  alimentes  en  air  comorime  et  courant  continu  par 
cordon  ombilical  d'environ  fl  m. 

AMRUA  :  la  commande  des  deolacements  du  cadre  est  confiee  a  un 
sequenceur  logique  programmable,  le  TRANSIMATE  (4),  dont  les  reacti¬ 
ons  se  fondent,  non  sur  I'etat  des  entrees  logiques,  mais  sur  les 
transitions  d'un  etat  bas  a  un  etat  haut  (0  et  5  \l)  de  ces  entrees. 

La  carte  transimate  develoooee  est  reprise  en  figure  4:  elle  presente 
30  lignes  d'entree,  chacune  d'elles  oouvant  §tre  inhibee,  lues  suc- 
cessivement  (l  microsecondi/entree)  et  comparees  a  leur  etat  precedent 
par  un  circuit  "coeur":  en  fonction  de  l'etape  courante  et  de  la  tran¬ 
sition  observes,  16  fonctions  de  sortie  ou  3  temoorisateurs  oeuvent 
Stre  actives.  Cette  carte  n ‘utilise  oas  de  microorocesseur  mais  4 
EEPROMs  (outO,SEQ,outl, Timer)  dont  le  chargement  est  assure  a  partir 
d'un  PC  supoartant  un  programme  dont  1' execution  impose  simolement  a 
1 'utilisateur  de  remplir  4  tableaux  de  type  "etaoe-action" , 

Le  PC  suonarte  d 'autre  oart  le  programme  Pascal  traduisant  la  strategie 
A3£ d'evitement  d'obstacles:  deux  autres  cartes  electroniaues  ont  done 
ete  develoDoees  (5);  1'une, mantes  sur  le  PC, assure  la  generation 
d'imoulsians  vers  le  transimate,  la  commande  d'un  moteur  electrique 
suooortant  un  caotsur  exteroceotif  a  uitrasons  POLAROID,  et  la  corn- 
mande  du  cacoeur  iui-mgme;  1' autre,  montee  sur  le  robot,  assure  le 
traitement  des  signaux  en  adaotant  leur  niveau  aux  capteurs,  moteurs 
et  electrodistributeurs  utilises. 


Pin. 4  La  carte  Transimate 


Pin  6.  Schema  de  eontrdle 


ROMOCAN  :  tandis  aue  la  figure  5  donnait  le  schema  general  develoope 
oour  le  robot  orecedent,  la  figure  5  suivante  reorend  1 'architecture 
de  commands  de  ROMOCAN.  Aucun  seauenceur  lagiaue  n'est  utilise,  la 
commands  est  directement  confiee  au  PC  (programme  Pascal):  une  carte 
d 'entrees/sorties  FCP-024  de  Flytech  Technology  (s)  a  ete  utilisee  et 
montee  sur  le  PC,  tandis  que  deux  cartes,  une  reprenant  la  oartie  logi- 
que  des  circuits  caoteurs,  1 'autre  la  partie  puissance  des  moteurs 
et  electrodistributeurs ,  sont  montees  sur  le  robot.  Oes  capteurs  ooti- 
aues  incrementaux  rotatifs  d'une  precision  de  2°  accomoagnent  chaque 
moteur,  tandis  que  des  capteurs  fin  de  course  g&rent  la  translation 
du  cadre  intermediaire. 


Fjo  A.  Schema  de  contrOle  Ramacan 

Ce  dernier  robot  est  dong  un  vecteur  de  transport  d'une  camera  de 
surveillance  Ikeoami  ITC-410  de  faible  masse  (l,fl  kg);  le  premier 
est  surmonte,  lui,  d'un  caoteur  a  ultrasons  Polaroid  permettant  la 
detection  d 'obstacles  oar  calcul  de  leur  distance  au  robot  et  non 
par  identification. 

Le  oaragranhe  3  decrira  la  stratpgie  d'evitement  d'obstacles  aooliquee 
au  robot  AMRUA ,  tandis  due  lr  oaraaraohe  4  riecrira  1 'identification 
d'objectifs.  Cans  le  premier  cas,  le  caoteur  est  simple  car  1'accent 
repose  cl' abort!  sur  le  contrQle  intelligent  d'un  rooat  mobile  dant  les 
deolacements  sont  imposes  constants;  dans  le  second  cas,  1'accent 
repose  sur  le  traitament  des  siqnaux  ri'idsntif'ication  et  d'images  ou' 
un  rieolacement  du  robot  ou  une  rotation  du  robot  cermet  oventuellement 
d 'af finer. 
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3, 'Algorithms  si'  ^onliaue  %  la  -spbilite  intelligence. 

3.1,  Algorithms  A jK 

L'algorithme  A*  est  conrvu  et,oar  exemole, 
decrit  d-ns  (?,R,9).  Des  I 'instant  ou  \e  probleme  gere,  an  1 'occu¬ 
rence  I'evitement  d 'obstacles  oar  un  robot  mobile,  peut  &tre  modeli- 
se  oar  un  graphs  composes  de  noeuds  n  st  o  ' arcs  les  reliant  entre 
eux  et  assacj.es  a  une  fonction  coOt,  11  oeut  §tre  applique. 

3.1.1.  Modelisatton  de  1  'envirannement. 

Le  robot  AMRUA  est  souioe  d'un  caoteur  a 
ultrasons  monte  sur  un  moteur  a  cc  dont  une  rotation  incremental© 
peut  §tre  orogrammee.  Ce  caoteur  oresente,  raopelons  le,  trois  incon- 
venients:  l'intensite  sonore  riecrolt  comme  le  carre  de  la  distance 
et  est  attenuee  oar  la  viscosite  orovoquant  des  oertes  deoendantes 
de  1'humidite  de  l'air  et  de  la  frequence  de  l'cnde  transmise;  ses 
performances  sont  affectees  oar  la  vitesse  du  son,  done  dependent  de 
la  temperature  et  cie  la  vitesse  et  direction  du  vent;  la  largeur  du 
faisceau,  enfin,  le  "beam  angle"  depend  du  diametre  du  caoteur  et  de 
la  frequence:  cette  derniere  est  ennuyeuse  car  une  frequence  elevee 
reduit  la  largeur  du  faisceau  mais  augmente  1 'attenuation  du  signal  '• 
et  done  reduit  la  oartee.  Enfin,  deux  types  d'erreur  doivent  @tre 
traites:  l'erreur  de  distance  liee  au  "beam  angle"  riOe  au  fait  que 
la  distance  mssuree  est  la  distance  a  l'objet  le  olus  proche  (fig. 7) 
et  l'erreur  de  non-detection  dQe  a  la  reflexion  soeculaire  (fig.fi) 


Fig,  7  g. Desired  measurement  Fig,  P 


bi Actual  reading 


Ces  erreurs  peuvent  §tre  minimalisees  oar  repetition  de  mesures  sous 
differents  angles.  Elies  se  traduisent  neanmoins  par  un  traitement 
logiciel  plus  lent  et  une  commande  plus  soqhistiouee, 

Le  mode  de  representation  de  1 ' environnement  utilise  se  base  sur  une 
division  de  celui-ci  en  cellules  de  10  cm  de  cSte  (voir  dimensions  du 
robot)  qui  sont  soit  vides  (d'obstacles) ,  soit  occupies,  soit  inconnues, 
Lars  de  la  rotation  du  capteur,  chaque  "vision"  engendrera  un  triangle 
de  precisian  dont  1 'angle  au  sommet(  le  capteur)  est  impose  par  1 'in¬ 
crement  (7°)  et  la  hauteur  par  la  distance  a  1 'obstacle  le  plus  Droche 
ou  par  la  portee  maximale  orogrammee  (  4  a  5  m  dans  notre  cas). 
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Chaaue  triangle  de  precision  est  ensuite  traite:  les  cellules  interi- 
eures  aux  triangles  sont  vides,  les  cellules  interferant  avec  le 
oolygone  de  visibility  canstruit  a  Dartir  des  triangles  de  precision 
successifs  sant  occuoees  ou  iriconnues.  Pour  reduire  le  temns  de  calcul 
un  algorithme  recursif  simple  est  utilise:  partant  d'un  paint  cannu 
interieur  au  contour  et  d'une  discretisation  du  contour  sous  forme  de 
pixels,  1 ‘environnement  est  engendre:  exemple :figure  9.  ! 


Pip  P.  Carte  de  1 'environnement 


Lors  des  visions  ulterieures  f deplacements  du  robot),  la  carte  de 
1 1 environnement  doit  §tre  actualisee:  oour  chaaue  cellule,  on  garde 
en  memoire  son  etat  precedent,  la  distance  qui  la  seoarait  du  robot, 
l'etaoe  la  plus  recente  a  laquelle  elle  a  ete  traitee,  le  nombre  de 
fois  qu'elle  a  ete  vue  oleine  ou  vide. Acres  chaque  deplacement  du 
robot  (hyoothese  d' obstacles  mobiles),  une  nouvelle  vision  n'affecte- 
ra  l'etat  d’une  oelluie  que  si  sa  position  actuells  est  MOINS  distante 
de  celle  du  robot  que  lors  de  l'etape  orecedente. 


3.1.2.  Generation  de  trajectoires 


L'esoace  de  travail  du  robot  est  ramene 
a  une  grille  bidimensionnslls  de  ooints  reoartis  suivant  les  deux 
axes  X  et  Y,  Chaaue  oaint  represents  une  position  h  laquelle  le  robot 
oeut  se  rendre  et  constitue  un  noeud  du  graphe,  oour  autant  aue  les 
cellules  en  contact  avec  le  robot  place  en  (x,Y)  soient  vides  ou  incon- 
nues . 
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La  structure  du  robot  et  ses  quatre  deolacements  possibles  (x+,  X-, 

Y+,  Y-l  limitent  deja  le  nombre  de  noeuds  susceotibles  d'etre  generes. 
La  fonction  heuristioue  de  l'algorithme  A*  est  definie  oar: 

F'(n)  »  W  ,G'(.n)  +  Wh.H'Co) 

Si  Xn  st  Yn  representent  les  coordonnees  du  noeud  n  et  Xb,  Yb  celles 
du  noeud  "but"  prealablement  choisi,  H'(n)  representera  la  distance 

H'(n)  =  \/(*Xn-Xb)2  +  (Yn-Yb)2 

(en  fait,  cette  distance  est  fictive  dans  le  cas  du  robot  AMRUA  ne 
disposant  que  de  deux  degres  de  liberte) 

tandis  que  G'(n)  sera  une  estimation  (les  increments  de  deplacement 

sont  fixes  mais  sujets  a  imprecision)  de  la  distance  parcourue  du 

noeud  actuel  ou  courant  au  noeud  n.  W  et  W.  sont  des  coefficients 

9  h 

.  de  ponderation  oermettant  une  recherche  de  type  "breadth-first, W  =1; 
depth-first,’.'/^! ;  ou  best-first,  W^=W^"  si  Qn  minimise  F'(n).  ® 


Fig  ID.  Exemoles  de  tra,jectoires  engendrees. 
(',Vq=G,7  ;  V/h  a  1,5  ;  Dmax  a  4  m) 


/iL./  O 
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3.2.  Generation  de  sous-buts. 

Lorsoue  l'objectif  a  atteindre  est  "invisible" 
ou  le  devient,  il  est  necessaire  de  generer  un  sous-but,  dit  promet- 
teur  s'il  est  proche  d'une  frontiere  avec  l'inconnu  d'ou  le  caateur 
du  robot  oourra  proceder  a  un  balayage  susceptible  de  comoleter  sa 
carte  de  1 'environnement  et  de  voir  le  but  final.  Au  stade  actuel, 
bien  qu'an  commence  a  pressentir  l*inter§t  de  1 'introduction  ties  oroba- 
bilites,  un  sous-but  orometteur  sera  choisi  dans  la  lists  "CLOSED" 
(liste  des  noeuds  deja  traites)  triee  sur  une  valeur-tyoe  definie  par: 

D  :  un  displacement  a  partir  de  n  n'entraine  aucune 
collision. 

1  :  entraine  une  collision  avec  un  obstacle  seulement 

2  :  entraine  une  collision  avec  un  obstacle  et  l'inconnu 

3  :  entralne  une  collision  avec  le  seul  inconnu. 


Fio  11.  Valeurs-tyoes 


Un  sous-but  de  valeur-type  3  sera  orefere  aux  sous-buts  do  valeur-type 
inferieure, 

L 'algorithms  decrit  est  particuliarement  oerformant  en  site  couvert 
et  dans  1'hypothese  d 'obstacles  fixes  entre  visions  successives.  Nous 
entamons  a  present  1 'etude  mains  simole  de  la  mobilite  en  site  ouvert 
tanriis  nue  la  programme  resume  ci-dessus  et  entierement  decrit  dans  (10) 
est  proqressivement  complete,  emoiriouement,  sur  base  des  observations 
de  laboratoire. 
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4,  Reconnaissance  d ' ob  iectifs . 

4.1.  Ls  orobleme  de  la  reconnaissance. 

On  develoooe  ci-dessous  une  fagon  de  concevoir  la  construction  d' 
un  systeme  destine  a  la  reconnaissance  automatiaue en  infrarouge  ther- 
mique ,  d'objectifs  3-0  situes  dans  une  scene  comolexe  et  dont  l'asoect 
est  fonction  de  la  distance.  L'exoertise  nous  aoorend  aue  le  crocessus 
de  reconnaissance  est  essentiellement  fonction  oe  la  distance.  On  a,  . 
oour  cette  raison,  distingue  deux  cas  differents:  le  oremier  cas  con- 
cerne  les  images  dans  lesouelles  l'abjectif  oeut  §tre  decompose  en  un 
ensemble  de  primitives,  le  second  les  images  pour  lesouelles  cette  de- 
comoosition  est  imoossible.  On  entend  par  primitive  un  objet,  element 
de  l'objectif,  aui  ocssede  ses  craares  attributs  et  au'on  oeut  oecomoc- 
ser  en  d'autres  primitives,  ce  theoriquement  juscu’au  niveau  "atomipup" 
Un  char  de  combat  oeut  oar  exemole  @tre  decomoose  dans  les  primitives 
suivantas:  tourells,  canon,  train  de  roues,  chassis  et  moteur.  La  train 
de  roues  oeut  a  son  tour  Strs  considere  comma  comoose  de  roues,  ri'un 
crabotin,  d'une  chenille,  at  ainsi  de  suite.  Le  choix  oroorement  dit 
ae  la  methode  de  reconnaissance  est  considere  comme  un  processus  flou 
oour  les  raisons  suivantas: 

-  la  resolution  de  l'objectif  dans  l'image  est  essentiellement  va¬ 
riable  a  cause  ces  conditions  atmosohericues  et  da  la  resolution 
prcore  au  systeme  d'imagaria  utilise. 

-  les  methodes  de  reconnaissance  utilisees  sont  fonction  de  la  dis¬ 
tance  du  systame  da  arise  de  vua  a  l'objectif  et  laurs  Unites 

d( utilisation  ne  sant  oas  figees. 

La  tableau  2  ci-dssscus  mcntre  un  axemole  de  corresocndan'ce  entra  la 
notion  floue  de  distance  introduite  oar  l'exoert  et  les  observations 
coresoondantes . 


Concent  flou  oe  DISTANCE 

Description  de  l'objectif 

tres  longue  distance 

Points  chauris:  oas  de  reconnai¬ 
ssance  oossible. 

longue  distance 

forme  globale  be  l'abjectif 

distance  mayenne 

composition  de  l'objectifs  en 
primitives 

caurte  distance 

description  atamioue 

Tableau  2.  Exemole  de  corresoandance  entrs  la  notion  floue  de 
distance  et  la  description  de  l'objectif. 

4.2.  Choix  de  la  methode  de  reconnaissance. 

Puisoue  la  methode  de  reconnaissance  deoend  de  la  distance,  on 
attribua  a  chaaue  procedure  k  de  reconnaissance  une  fonction  d'aooar- 
tenance  x^(d)  aui  decrit  an  queloue  sorts  la  fiabilite  du  choix  k 
relativement  a  la  distance  d  entre  l'objectif  et  le 
systeme  de  arise  de  vues. 
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Figure  12.  Exemole  de  fanctions  d 'aaoartenance. 

La  figure  12  sst  un  exemole  de  subdivision  de  l'echelle  das  distances 
oar  1 'intsrmediaire  des  differentes  fonctions  d 'aaoartenance  et  des 
■"ethodes  de  traitement  corresoondantes .  Oans  ce  oui  suit,  on  ns  oresen- 
tera  aue  les  methades  relatives  k  la  reconnaissance ,  gad  oour  des 
distances  camera-od jectif  courtes ,  mayennes  ou  longues  (tableau  2). 

4.3.  Reconnaissance  a  courts,  -ove-'nq  °t  longue  distance. 

4.3.1 . L'abicctif  oeut  9trs  decomnns?  ?n  ensemble  de  primitives. 

II  s'agit  des  conditions  oarticuliSres  ou  il  est  oossible  de  faire 
la  distinction  entre  differentes  orimitives  de  l'objectif  dans  une 
n Sine  image.  L'objectif  aeut  Stre  vu  comma  un  objet  tridimensionnel  de- 
crit  oar  les  orimitives  et  leurs  caracteristiaues  nroores.  II  ne  faun 
neanmoins  oas  nerdrs  de  vue  aue  le  svst^me  a  construire  est  destine  a 
traitor  les  imanes  ae  sortie  hidi^ensionnelles  do  systems  df observation, 
IJne  solution  au  orobl-Yme  is  reconnaissance  oeut  £trs  trouvce  dans  le 
dFveloooement  d'un  systame  exnert  relativement  simole  ( 1 X ) 

4. 3. 1.1.  Construction  do  nodeles  3-3 

La  oremiere  etaoe  de  la  methode  consists  a  construire  une 
banque  de  dannees  contenant  la  desqrintion  des  modeles  3-0  des  abjec- 
tifs.  Css  modeles  song  constitute  de  orimitives  hierarchisees  en  fcinc- 
tion  de  la  distance,  de  carte  au'!  une  classe  de  distances  donnee, 
relative  a  une  faction  d ' aaoartenance  soecifique,  (voir  la  notion  de 
function  a 'aooartenance  decrite  ci-avant)  corresoondent  des  orimitives 
de  l'objectif  ou'il  est  Possible  de  distinguer  a  la  distance  consideree. 
Les  figures  13  et  14  donnent  des  exemoles  de  relations  entre  orimitives. 
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Figure  13. :  exemple  de  graphe  geometrique 


Figure  14.  :  exemple  d  un  graphe  semantiquc  dc  type 


4, 3. 1.2.  Recherche  de  la  meilleure  projection  2-0 

La  seconde  etaoe  consists  *J  rechercher,  oam  toutes  las 
n rejections  oossibles  das  mndales  dans  la  olan  imacjB,  celle  oui  corres 
oprd  le  siieux  avec  1  'i^aqs— sortie  du  systewe  d 'observation,  Cette  re¬ 
cherche  neut  Stro  ootinalisee  an  tenant  comets  de  rooles  oarticulieres 
tirees  r*e  la  connaissance  nriori  aue  I'on  a  de  X'objectif  et  de  la 
scene, 

a, 3,1.?.  Caracteristioues  do  la  solution. 
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-  Un  traitement  prealaole  des  images  est  souvent  necessaire  pour  redui- 
re  le  bruit  (surtout  en  infrarouge  thermique) 

-  Un  ob.jectif  peut  §tre  reconnu  a  oartir  d'un  nombre  limite  de  primitives 

-  II  est  indispensable  de  construire  une  base  de  connaissance 
relativement  elaboree  et  comolexe. 

-  La  methode  permet  une  segmentation  dynamique  de  I'image,  a  cette 
intention,  on  attribue  une  methode  de  segmentation  oarticuliare  a 
chaque  primitive. 

-  La  recherche  d'une  primitive  oarticuli&re  dans  I'image  est  un  pro¬ 
cessus  local  et  est  done  limitee  a  une  petite  partie  de  I'image. 

4.3.2.  L 'ob.jectif  ne  oeut  oas  gtre  decomoose  en  primitives. 

Dans  ce  cas ,  seule  une  forme  globale  de  l'objectif  est  disoanible  sur 
I'image,  et  une  methods  globale  de  reconnaissance  des  formes  s'innose 
souvent.  ’.La  maniere  classiaue  de  oroceder  consiste  a  effectuer  tout 
d'abord  une  segmentation  de  I'image,  ensuite,  a  determiner  ies  carac¬ 
teristiques  nroores  f features)  de  chacune  des  ref inns  obtenues  et  enfin 
a  ooerer  une  classification  aptimale  de  ces  regions  au  sens  d'un  cri- 
tere  particulier.  Une  methode  moins  classiaue  consiste  h  aoolicuer 
directament  au  indirectement  aux  donnees  des  methodes  basses  sur  les 
modeles  connexionnistes ,  aooeles  aussi  reseaux  de  neurones  artificiels 
ou  encore  memoires  associatives . 

4. 3. 2.1.  Methodes  classiques. 

LA  SEGMENTATION 

Le  traitement  orealable  a  tout  autre  consiste  dans  ce  cas  .i  ooerer  une 
partition  de  I'image  en  sous-imaqes  au  regions,  eventuellement  disjoin- 
tes  qui  idealement  corresoondant  aux  dif^erents  objets  aui  comoosent 
I'image,  On  citera  oar  exemole:  la  segmentation  car  seuillage  des  niveaux 
de  gris,  segmentation  oar  croissance  de  regions,  segmentation  oar 
regrauoement  de  oixels  f clustering) ,  segmentation  car  transformee  de 
distances,  segmentation  oar  regrauoement  de  regions,  segmentation  oar 
modeles  de  Markov  caches,  etc. 

LA  RECHERCHE  DES  CARACTERISTIQUES  DES  REGIONS 

Cette  operation  fait  logiauement  suite  h  la  orecedente  et  asut 
cansister  o  rechercher  des  grandeurs  caracteristiques  et/nu  descri- 
otives  d'une  reqion  dans  le  but  de  la  distinguer  des  autres  (discri¬ 
mination!  nt  ulterisuerement  de  la  reconnaltre  comme  objet  ou 
oartis  d 'objet  (identification).  On  citera,  \  titre  indicatif, 
queleaues  caracteristiques  fort  utilisees,  a  savoir: 

-  les  mqments  qeometriaues  orincioaux  du  premier  et  du  second 
ordre,  qui  off rent  1'avantage  d'6tre  invariants  en  translation 
et  en  rotation. 

-  les  grandeurs  caracteristiques  de  la  texture  (l2) 

-  la  description  des  cnntours,  nar  exemple  sous  forme  d'une  suite 
de  cauoles  (longueur,  orientation)  au  encore  d'une  suite  de  codes 
resultats  de  1  'application  d'une  grammaire  oredefinie  (13), 

-  etc. 

LA  CLASSIFICATION 

Cette  aneration  consiste  a  classer  le  plus  carrecotement  possible 
les  regions  decrites  oar  leurs  attribute  et  obtenues  lore  des  etaoes 
nrecedentes  (id). 


'■ ' -.-  '  ;.A  .  .  ,.  ;-_■-  ,  -  .  ...j  ■.  '  ■  ■■ .  .  ■  • ,••  ■  .  -v-  •..•••  .;•..  .  • .-  -;*••■■:• ■■  ■_■  "’ iry>  --P^.f  ■:A<^ 
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4. 3. 2. 2.  Methades  utilisant  les  reseaux  CDnnaxionnistes. 


Les  reseaux  connexionnistes  sont  composes  d1 elements  de  calcul  non- 
lineaires  elementaires  (voir  figure  15)  dont  1 'entree  est  la  somme 
oanderee  des  signaux  d 'entree  et  dont  la  sortie  est  generalement  la 
signature  (-l,+l)  de  la  somme  abtenue.  Ces  elements  de  calcul  sont 
organises  en  couche,  de  sorte  oue  chague  element  d'une  couche  est 
relie  a  tous  les  elements  de  la  couche  precedente  (voir  figure  15) 


pops 


Neurone  aitifieiel 

Figure  15. ;  element  de  calcul  elementaire 


Figure  16. :  Organisation  en  couches 
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On  oresente  tree  brievement  ci-dessous  un  reseau  particulier  capable 
d'aoprendre  oar  lui-mgme  et  aui  a  eta  developpe  oar  S.Grossberg  f  15 ) : 
il  s'agit  des  mcdeles  ART  (Adapatative  Resonance  Theory)  decrits  a  la 
figure  17. 


Figure  17.  :  reseau  ART  (Adaptive  Resonance  Theory) 


l ’information  antrante  (une  image  et/ou  ses  attributs)  est  transmise 
\  la  cauche  FI  et  ast  compares  i  1 ‘information  deja  memorisee  orece- 
demment  at  reoresentee  oar  un  neurone  ds  la  coucns  F2.  Si  la  difference 
entre  las  deux  informations  (a  savoir  celle  de  1 'entree  et  cells  deja 
memorisee)  est  jugee  nogligeable  o=;r  un  organs  de  cantr&le,  le  systems 
admet  aue  1 'information  d'sntree  correspond  a  celle  deja  memorisee  oar 
la  neurone  de  la  couche  F2  interrooe  et  adaote  ses  connaissances  en 
adaotant  le  ooids  des  connexions  ontre  couches,  Au  contraire,  si  1' 
organs  de  controls  June  aue  les  deux  informations  ne  correspondent  oas, 
la  systsme  desactiv/e  le  neurone  intsrroge  de  la  cauche  F2  et  essaie 
le  ouivant,  Si  ri1 -venture,  aucun  neurone  de  F2  ne  satisfait,  le  syste- 
me  area  une  nnuvelle  "classe  d 'information"  en  creant  un  nouveau 
neurone  dans  la  couche  F2 .  Lornnue  las  grandeurs  d 'entree  sont  loginues, 
le  reseau  est  anoele  ART1 ,  lorsque  cas  grandeurs  sent  analogicues  le 
reseau  est  aaoele  ART2 . 

Dans  le  cas  de  ART1 ,  1 'entree  du  systome  est  constitue  oar  des  images 
binaires  obtenues  oar  sauillaqe  des  images  resultats  du  filtrage  des 
images  original ss  oar  un  f litre  aasse-naut  destine  %  Pairs  ressortir 
las  contours, 

Dans  Is  cas  ds  ART2 ,  1 'entree  du  systSme  est  constitute  oar  les  images 
origxnalos , 

Lee  fitments  d'ingae  de  la  figure  1R  rcoresentent  le  type  d'pbjectifs 
concernes.  Les  resultatr  obtenua  dans  les  deux  cas  sont  tres  oromet- 
teurs  flS^ 

a, 3, 2. 3.  Garactf ristinuos  mas  methods*  connexiannistss  ART, 
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Figure  18  :  (a)  image  binairc  presentee  a  ART1  (b)  image  presentee  a  ART2 


-  un  pretraitement  des  images  d 'entree  est  generalement  indispensable 
du  fait  que  ces  methodes  sant  sensibles  a  des  translations,  des 
rotations,  des  changsments  d'echelle  et  des  variations  de  brillance. 

-  si  le  pretraitement  dont  auestion  ci-dessus  conserve  1 'interpretation 
visuelle  de  1' image,  il  est  passible  de  visualiser  1' information 
memorisee. 

-  1 'aoorentissaqe  est  adaatatif,  on  distingue  en  effet  a  ce  stade  deux 
sertes  d 'aoorentissaqe:  1 'aoorentissaqe  long  terms,  qui  concerne  la 
creation  de  nauveaux  neurones  et  de  nouvelles  connexions  entre  couches 
et  1 'aonrentissage  court  terme  aui  concerne  l'adaotation  des 
connexions  entre  couches. 


5.  Conclusions. 

La  nrogressian  de  robots  mobiles,  type  chars  automatises,  sur  le 
champ  de  bataillss  terrestre  imnase  d'abord  oue  soient  resolus  les 
orohlemes  relatifs  \  la  mice  au  naint  de  caoteurs  exteroceotifs 
nerformants  f fonctxon  distance,  conditions  atmosgherinues.etc. ) , 
au  traitement  des  sicnaux  recueillis  f impulsions ,  images,  fanction 
memorisation  associative  au  non),  i  la  generation  de  decisions 
( evi tenant  d'obstacles,  fir  sur  objectifs  reconnus ,  etc.).  Ces 
etudes  neuvent  Stre  nenees,  dans  une  mesure  importante  et  a 
foible  investissqment,  i  nartir  de  robots  mobiles  de  dimension 
modeste  evoluant  dans  un  envirnnnement  oroqressivement  plus 
camnlexe.  L 'architecture  definitive  ou  semi-definitive  de  tels 
systemes  robatises  maciles  denendra  lorgement  de  la  resolution 
des  oroblemes  precedents  (car  exemnle,  la  raoidite  de  deolacement 
ou  le  caractSre  cpntinu  ou  discret  des  deplacements  depend  de 
la  raoidite  de  traitement  des  signaux  recueillis):  raison  pour 
lciQuolle  nous  ernycins  nu'il  faut  investir  de  fagan  selective  dans 
la  recherche  "robotioue"  en  orivilCgiant  d'abord  l'asoect  syst&ne- 
exoert. 
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Photo  1:  AMRUA  (non  couvert) 


Photo  P;  Systeme  interne  de  baiayage 
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1.0  INTRODUCTION 


1.  The  main  characteristic  of  military  operations  are  totally  different  than  the  tasks  carried 
out  by  industrial  robot  in  a  factory  environment.  The  reason  for  this  is  the  nature  of  environment 
in  both  operations.  Military  operations  take  place  in  an  unstructured  and  uncertain  environment 
which  can  not  be  controlled.  Therefore,  all  sophisticated  techniques  of  programming  robots  to  carry 
out  task  repetitively  in  a  structured  environment  will  not  simply  work  for  military  operations. 
Consequently,  the  only  solution  for  this  problem  is  some  form  of  autonomy  under  the  supervision  of 
an  operator.  To  begin  with,  teleoperation  has  been  used  extensively  in  a  range  of  applications  to 
understand  the  remote  manipulation  concept  and  human-system  interactions.  It  also  provided  a  basis 
for  a  smooth  transition  towards  a  higher  autonomy  in  a  robotic  system. 

2.  A  robotic  system  which  must  cope  with  an  unstructured  environment  should  deal  with  the 
problems  provided  in  Figure  1.  A  fully  autonomous  robot  will  be  responsible  to  deal  all  of  these 
problems  [1],  However,  the  ability  of  robots  to  perform  tasks  autonomously  under  sensor-based 
control  is  rather  limited  and  human  intelligence  can  be  quite  useful  to  provide  a  complete  system 
solution  for  military  operations.  Considering  the  state  of  the  art  of  robotics  technology,  the  robot’s 
best  abilities  are  essentially  at  the  execution  level  such  robot  manipulation,  while  the  least  capabilities 
are  at  the  decision  level  of  the  task  hierarchy  where  the  operator  excels. 

3.  Among  the  desired  features  for  robot  manipulation  for  extending  the  field  of  future 
applications,  are  autonomous  trajectoiy  generation  and  automatic  collision  avoidance  with  known  or 
unexpected  objects  in  the  work  volume.  The  artificial  mechanical  impedance  method  analyzed  in  this 
paper  is  based  on  the  modulation  of  the  torques  in  such  a  way  as  to  produce,  between  the  target  and 
the  manipulator  endpoint,  the  dynamic  effect  of  a  desired  linear  or  nonlinear  mechanical  impedance. 
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Figure  1:  Robotic  Activities  anti  Functional  Distribution 
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Figure  1;  Robotic  Activities  anil  Functional  Distribution 
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The  result  of  using  this  method  is  the  integration  of  the  controller  model  with  the  nonlinear  dynamics 
model  of  the  robot  into  a  single  mechanical  model  which  is  used  to  obtain  the  desired  mechanical 
properties.  A  particular  feature  of  the  method  is  that  the  generation  of  a  trajectory  resuits  ’naturally’ 
from  certain  Cartesian  space  specifications. 

4.  In  this  paper,  the  artificial  impedance  method  is  used  for  the  trajectory  generation  of  a 
jointed  manipulator  in  free  motion  and  for  the  trajectory  correction  in  order  to  avoid  collisions  with 
obstacles.  Simulation  results  for  a  two  degree-of-freedom  manipulator  are  compared  with  those  of 
a  PD  position  controller.  For  obstacle  avoidance  the  artificial  impedance  approach  is  used  in 
conjunction  with  a  ’coastal  navigation’  scheme  which  provides  a  means  of  generating  trajectories  in 
non-convex  spaces  utilizing  proximity-sensor  data.  The  technique  has  the  potential  of  permitting 
operation  within  an  unknown  workspace  and  of  providing  trajectory  correction  to  avoid  collisions  with 
unexpected  or  moving  objects.  Simulation  results  illustrate  this  technique. 

5.  Artificial  impedance  approach  of  a  single  arm  motion  control  has  been  intensively  analyzed 
in  the  last  decade.  The  results  of  the  first  part  of  the  paper  show  that  the  trajectory  of  a  single  arm 
robot  can  be  generated  in  real  time  by  assuming  artificial  impedances  in  the  robot  work*'  'lume.  In 
the  second  part  of  the  paper,  artificial  impedance  approach  is  applied  to  dual  arm  robots. 

6.  Dual  arm  robot  motion  is  analyzed  for  two  distinct  phases,  the  approaching  phase,  when 
each  arm  is  moving  independently  toward  the  desired  positions,  and  the  coordination  phase,  when 
the  two  arms  are  transporting  together  an  object  to  a  desired  position.  The  motion  in  the 
approaching  phase  will  be  coordinated  by  introducing  artificial  attractive  impedances  between  each 
arm  and  the  desired  positions  (to  generate  attractive  forces  between  the  arms  end  points  and  the 
targets)  and  artificial  repulsive  impedances  between  the  arms  and  obstacles  (to  generate  artificial 
repulsive  forces  and  avoid  the  collisions  of  the  moving  arms)  in  the  case  of  the  detection  of  obstacles 
on  the  arms  trajectories. 

7.  The  motion  in  the  second  phase,  when  the  two  arms  are  holding  a  rigid  object,  represents 
the  motion  of  a  closed  loop  mechanism.  .Artificial  impedance  approach  to  motion  coordination,  in 
this  case,  is  based  on  designating  one  arm  as  a  leader  arm  and  the  other  as  a  follower  arm.  In  order 
to  move  the  piece  safely,  materials  and  holding  requirements  impose  kinematics  and  dynamics 
constraints  at  the  contact  points  of  the  piece  and  the  two  arms.  When  using  artificial  impedance 
approach,  these  constraints  are  transformed  into  virtual  impedances  between  the  two  arms  which 
simulate  maximum  elastic  forces  and  torques  applied  on  the  piece  and  maximum/minimum  holding 
forces.  This  approach  permits  coordination  of  the  motion  of  the  two  arms  in  the  same  fashion  as  in 
the  case  of  free  motion  by  introducing  suitable  artificial  impedances  between  the  arms  and  the  desired 
positions  and  the  obstacles. 

8.  The  paper  presents  the  analytical  model  of  the  impedance  based  control  of  a  dual  arm 
robot,  as  well  as  simulations  of  the  operation  of  the  impedance  based  controller  in  the  approaching 
and  coordination  phases  of  the  motion  of  the  two  arms.  The  objects  will  be  assumed  in  simulations 
as  rigid  bodies,  elastic  bodies  and  as  strings. 
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9.  In  the  last  part  of  the  paper,  implementation  aspects  of  the  artificial  impedance  approach 
are  analyzed.  In  particular,  various  nonlinear  compensation  methods  of  linearization-decoupling  of 
the  robot  are  reviewed.  The  state-feedback  compensation  method  is  compared  to  the  perturbation 
observer  method.  Robustness  aspects  are  discussed  and  a  Three-part  control  scheme  is  presented. 


2.0  MILITARY  REQUIREMENTS  FOR  MANIPULATOR  CONTROL 

10.  Various  military  organization  are  investigating  the  roles  of  robotics  in  military  operations. 
It  is  commonly  accepted  that  the  robotics  technologies  will  significantly  affect  the  issues  such  as 
manpower  requirements  (i.e.  reducing  the  number  of  personnel,  increase  individual  effectiveness), 
and  soldier  risk  exposure  (i.e  survivability)  in  all  types  of  warfare  including  nuclear,  biological, 
chemical  weapons  and  as  well  as  in  peace  keeping  operations. 

11.  In  future,  military  robots  will  be  capable  of  operating  autonomously  on  the  battlefield. 
Such  an  advanced,  intelligent  robot  would  be  able  to  take  place  of  a  soldier,  to  move  in  difficult 
terrain,  tc  recognize  dangers,  distinguish  between  friend  and  enemy,  and  fight  very  efficiently  with 
minimum  human  interaction.  However,  the  state  of  the  art  of  robotics  technologies  show  that  it  is 
currently  not  possible  to  have  fully  autonomous  robots  to  meet  military  requirements.  Therefore,  an 
effective  solution  for  military  operations  is  the  limited  autonomy  or  supervised  autonomy  where  the 
responsibilities  are  distributed  according  to  capabilities  between  the  operator  and  the  robotic  system. 
The  general  activities  and  functional  distribution  are  shown  in  Figure  1. 

12.  In  order  to  use  robots  effectively  in  military  operations  (considering  the  risk  associated 
with  the  sensitivity  of  the  tasks  in  military  operations  dealing  with  explosive  materials,  etc.),  the  robot 
controller  system  should  have  extremely  robust,  reliable  and  accurate  manipulation  capabilities. 
Among  the  desired  features  for  robot  manipulators  capabilities  in  future  applications  are  autonomous 
trajectory  generation  and  automatic  collision  avoidance. 


3.0  FORECAST  OF  VARIOUS  APPROACHES  TO  ROBUST  MANIPULATION 
CONTROL 


13.  Cartesian  space  control  formulation  of  robot  manipulators  led  to  the  development  of 
artificial  potential  field  and  artificial  impedance  approaches.  Continuum  mechanics  formulation  of 
the  artificial  potential  field  consists  in  choosing  an  attractive  potential  field  to  correspond  to  the 
potential  field  of  linear  elastic  springs  while  the  repulsive  potential  fields  are  chosen  to  correspond 
to  potential  fields  of  nonlinear  force  functions  dependent  on  the  shortest  distances  between  the 
manipulator  parts  and  each  obstacle. 

14.  Khatib  extends  the  Lagrangian  formalism  of  robot  dynamics  to  include  artificial  potential 
fields  (APF)  involving  an  attractive  force  field  associated  with  the  end  effector  and  the  position  to 
be  reached,  and  fields  of  repulsive  forces  associated  with  obstacles  and  the  manipulator  parts  [2], 
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15.  The  continuum  mechanics  formulation  requires  equations  which  provide  a  continuum 
geometric  modelling  of  the  obstacles  using  primitives  such  as  points,  lines,  planes,  ellipsoids, 
parallelipipeds,  cones  and  cylinders.  Also,  as  in  all  artificial  potential  field  and  impedance  approaches 
to  Cartesian  control  of  the  robot  motion,  the  robot  arm  dynamics  as  seen  by  the  robot  controller  is 
decoupled  by  compensation  of  the  inertial,  centrifugal,  Coriolis,  gravitational  and  other  nonlinear 
components  of  the  dynamic  equations.  In  addition,  the  APF-based  controller  needs  a  dissipative 
force  term  in  Cartesian  space  added  to  obtain  asymptotic  stability. 

16.  Takegaki  and  Arimoto  [3]  develop  a  joint-space  attractive  potential  field  in  the 
Hamiltonian  formalism  for  conservative  forces.  The  Hamiltonian  in  this  case  represents  the  total 
energy  of  the  system,  which  is  used  as  a  Lyapunov  function  for  p.oving  asymptotic  stability.  Any 
desirable  potential  function  which  is  positive-definite  may  be  chosen.  A  linear  feedback  of  joint 
velocity  is  added  to  the  joint-torque  control  law  for  asymptotic  stability. 

17.  Andrews  and  Hogan  [4]  simulate  the  avoidance  of  a  moving  target  using  the  APF 
approach.  They  show  also  that  the  attractive  artificial  potential  field  is  equivalent  to  artificial 
mechanical  impedances  in  Cartesian  space  between  the  manipulator  end-effector  and  the  target  for 
the  case  of  conservative  fields.  A  dissipative  term  in  the  control  law  is  added  for  stability. 

18.  Slotine  [5]  continues  the  joint-space  stability  analysis  of  Takegaki  and  Arimoto  including 
directly  the  damping  part  of  the  joint  control  law  in  the  power  balance  equations  of  the  manipulator 
dynamics  and  joint  actuators.  In  the  Hamiltonian  formalism,  for  conservative  forces  the  derivative 
of  the  total  energy  (the  actual  manipulator  and  the  virtual  energy  associated  with  the  proportional 
part  of  the  joint  controllers)  is  known  to  equal  the  power  dissipated  by  nonconservative  forces, 
namely,  the  artificial  joint  dampers  correspond  to  the  damping  component  of  the  joint  control  law. 
The  overall  approach  to  control,  while  not  formulated  for  Cartesian  space,  provides  an  interesting 
background  for  interpreting  .APF  and  Artificial  Impedance  approaches. 

19.  Krogh  [6]  introduces  discontinuous  generalized  potential  fields  which  surround  an  obstacle 
up  to  a  chosen  distance  from  the  obstacle  and  are  defined  as  position  and  velocity  dependent 
potentials.  The  resulting  generalized  potential  fields  cannot  be  interpreted  in  terms  of  artificial 
mechanical  impedances. 

20.  A  time-optimal  approach  to  a  robot  controller  based  on  generalized  potential  fields  is 
simulated  by  Newman  and  Hogan  [7],  The  resulting  bang-bang  control  might  be  undesirable  for 
robot  control,  particularly  in  structurally  flexible  systems,  because  of  the  possibility  of  exciting  high 
order  (probably  unmodelied)  dynamics.  Cartesian  decoupling  is  a  prerequisite  for  the  success  of  this 
scheme. 

21.  The  .APF  control  schemes  discussed  above,  [2]  to  [7],  are  formulated  as  continuum 
mechanical  problems  and  so  require  equations  describing  the  geometric  shapes  of  objects  in  ti.c  robot 
workvolume  in  continuum  mechanical  form.  The  actual  application  of  these  schemes  is  based  on 


C13.7 


AC/243-TP/3 


lumped  parameter  control  laws  which  suggest  that  it  is  perhaps  more  appropriate  to  consider  a 
lumped  parameters  formulation  of  the  dynamics  in  robot  Cartesian  space  and  artificial  mechanical 
impedances. 

22.  Hogan  [8], [9],  presents  a  rigorous  formulation  of  the  Artificial  Impedance  approach  in 
robot  Cartesian  space.  He  applies  it  to  free  motion  and  to  object-contact  motion  of  a  robot  arm. 
Joint  torque  control  laws  are  derived  for  realizing  a  desired  diagonal  artificial  mechanical  impedance 
between  the  end  effector  and  the  target.  This  scheme  does  not  use  inverse  kinematics,  but  requires 
nonlinear  decoupling  in  Cartesian  space  using  position  and  speed  measurements  in  the  joints  as  well 
as  object-contact  force  measurement.  A  stable  motion  results  in  the  experiments  presented  with  a 
two  degree  of  freedom  direct  drive  horizontal  robot.  No  collision  avoidance  experiments  are 
included. 

23.  None  of  the  foregoing  schemes  include  representation  of  the  contacted  object  dynamics. 
Kazerooni  [10]  develops  an  artificial  mechanical  impedance  approach  in  which  the  desired  position 
command  is  modified  oy  an  artificial  admittance  which  compensates  the  robot  compliance  for  the 
contact  force.  Anderson  and  Spong  [14],  present  a  hybrid  admittance/impedance  control  combining 
hybrid  position/forcc  control  and  artificial  impedance  approach. 

24.  Lawrence  [11]  analyzes  the  .Artificial  Impedance  approach  as  a  torque-command  scheme 
and  as  a  position-command  scheme  (when  computation/communication  delays  affect  the  impedance 
controller)  and  finds  the  two  schemes  complimentary. 

25.  Necsulescu  et  al  [12]  simulate  collision  avoidance  of  multiple  and  moving  obstacles  for  a 
two  degree  of  freedom,  planar  robot  equipped  with  a  controller  using  Artificial  Impedance  approach. 
The  results  show  that  trajectory  control  and  collision  avoidance  can  be  realized  using  the  Artificial 
Impedance  approach  and  provide  evidence  that  this  technique  is  a  promising  alternative  to  the 
preplanning  schemes,  as  is  proposed  in  numerous  other  papers,  for  example  [13]. 

26.  Dual-arm  robot  motion  control  requires  a  higher  level  of  controller  complexity  compared 
to  the  single-arm  robot  motion  control  because  of  the  need  to  continuously  coordinate  the  motion 
of  the  two  arms. 

27.  Various  combinations  of  position  and/or  force  control  of  the  two-arm  motion  have  been 
reported  in  the  literature.  Luh  and  Zheng  propose  a  leader/follower  coordination  strategy  with  no 
force  feedback  [22], [23],  Kinematics  constraints  for  the  closed  kinematic  chain  are  t  ed  for  the 
determination  of  the  follower  motion  given  the  loader  motion  in  terms  of  positions,  vci’nctries  and 
accelerations.  One  rigid  body  and  two  articulated  rigid  bodies  arc  used  as  loads.  The  dynamic 
dependence  of  the  leader  and  follower  input  generalized  forces  are  also  derived.  The  major  difficulty 
in  using  these  results  to  coordinate  the  motion  comes  from  the  lack  of  contact/force  feedback. 
Consequently,  such  a  coordination  scheme  would  be  successful  only  in  the  case  of  no  servoing  and 
modelling  errors. 
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28.  Ozguner  et  al  develop  an  approach  for  canceiling  the  cross  coupling  terms  of  the  dual-arm 
dynamics  which  reduces  the  control  problem  to  controlling  independent  joint  servomotors  [24].  This 
is  achieved  by  a  sliding  mode,  discontinuous  controller  with  gains  obtained  for  a  given  bound  on  the 
norm  of  cross-coupling  interactions.  Simulations  are  performed  for  free  and  contact  motion  of  the 
two  arms  with  no  load.  No  comments  are  made  concerning  the  evaluation  of  the  norm  of  cross¬ 
coupling  interactions  and  on  the  operation  for  various  loads. 

29.  Swern  and  Tricamo  propose  a  modification  of  the  position  controllers  of  the  two  arms 
intended  to  compensate  the  constraint  forces  applied  by  the  arms  on  the  load  [25].  This  is  achieved 
by  correcting  the  reference  positions  based  on  the  end-effector’s  measured  forces/torques  and  the 
load  stiffness  matrix.  A  flexible  rod  and  a  beam  are  used  as  loads  in  simulations  and  a  recursive 
identification  scheme  is  used  for  the  load  stiffness  matrix.  While  this  approach  uses  the  actual  load 
stiffness  matrix,  Kazerooni  and  Tsay  propose  an  artificial  compliance  for  a  similar  control  scheme  [26]. 
The  effect  of  joint  flexibility  on  the  steady-state  error  in  the  object  position  is  analyzed  by  Ahmad 
and  Guo  for  arms  with  position/force  controllers  [27]. 

30.  Some  dual-arm  motion  hybrid  coordination  schemes  are  developed  based  on  the 
workspace  force,  velocity  and  position  vectors  defined  as  symmetric  functions  on  their  joint-space 
counterparts  for  the  two  arms.  Uchiyama  and  Dauchez  propose  a  hybrid  control  scheme  for  a  chosen 
set  of  relative  and  absolute  force  and  velocity  in  the  workspace  [28].  Kopf  and  Yabuta  compare, 
experimentally,  the  leader/follower  and  hybrid  control  schemes  [29].  In  the  former  control  scheme, 
the  leader  is  position  controlled  and  the  follower  is  force  controlled  while  in  the  latter  control 
scheme,  absolute  position  of  the  centre  of  the  object  and  the  relative  (internal)  force  applied  on  the 
object  are  controlled  symmetrically  for  the  two  arms.  The  leader/follower  scheme  has  performed 
poorly.  This  can  be  explained  by  the  lag  in  the  reaction  of  the  follower.  The  hybrid  scheme  has  had 
restricted  use  because  of  the  requirement  of  the  symmetric  functions  mentioned  above. 

31.  Suh  and  Shin  generalized  the  leader/follower  scheme  by  allowing  the  follower  grasping 
position  to  change  when  reaching  a  configuration  singularity,  or  for  obstacle  avoidance  [30]. 

32.  The  literature  review  has  shown  that  the  dual-arm  motion  coordination  results  to-date 
cover  only  a  limited  variety  of  types  of  payloads  and  do  not  include  such  payloads  as  a  string/blankct 
or  a  vibrating  structure  (with  the  exception  mentioned  above). 

33.  Also,  collision  avoidance  for  dual-arm  robots  has  not  been  analyzed  in  detail,  but  only  the 
use  of  the  spatial  planning  approach  from  single  to  dual  arms  [31], 

34.  Several  solutions  were  proposed  for  robust  robot  control.  Some  Solutions  are  applicable 
to  the  Artificial  Impedance  approach  as  for  example  Model  Reference  Adaptive  Control  [38.39,40] 
and  Second  Method  of  Lyapunov-based  designs.  Model  Reference  Adaptive  Control  can  be 
formulated  for  a  reference  model  which  represents  a  set  of  three  independent  translational 
mechanical  impedances  and  three  independent  rotational  mechanical  impedances  i.e  six  second  order 
linear,  constant  parameter  differential  equations.  The  error  between  the  reference  model  positions 
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and  Cartesian  end-effector  positions  would  be  used,  in  this  case,  for  the  adaptation  mechanism.  This 
approach  benefits  from  the  robustness  of  MRAC  approach,  but  only  asymptotically  the  robot  arm 
would  behave  like  the  desired  Cartesian  space  impedances. 

35.  Second  Method  of  Lyapnuov-based  design^  has  been  used  for  obtaining  sliding  mode 
controllers  for  single  [41]  and  dual  arm  robots  [23].  The  application  of  a  sliding  controller  for 
implementing  an  Artificial  Impedance  approach  has  been  explicitly  formulated  by  J  J.  Slotine  [42,  pp. 
214-215],  In  this  case,  sliding  surfaces  are  chosen  as  linear  functions  of  the  errors,  for  the  Cartesian 
state  variables,  and  of  the  environment  contact  forces.  In  sliding  mode,  the  controller  would  make 
the  robot  behave  like  the  desired  impedance.  The  resulting  discontinuous  control  law,  inherent  to 
the  application  of  the  second  method  of  Lyapunov,  can  be  approximated  by  a  boundary  layer-based 
continuous  control  law.  Maintaining  sliding  mode  required  sufficient  torque  output,  large  actuator 
bandwidths  and  fast  closed  loop  components  other  than  the  robot  arm. 

36.  Second  Method  of  Lyapunov  can  be  applied  directly  for  obtaining  a  corrective  term  added 
to  a  linear  control  law  in  order  to  compensate  for  the  errors  resulting  from  inexact  cancellation  of 
the  nonlinear  dynamic  terms  by  nonlinear  state  feedback  compensator.  Again,  a  discontinuous 
corrective  term  would  result  and  a  continuous  approximation  solution  is  proposed  for  joint  decoupling 
case  [19,  pp.  227-236].  A  similar  formulation  can  be  developed  for  Cartesian  decoupling  case  and 
.Artificial  Impedance  approach  case.  An  interesting  alternative  in  this  case,  to  the  use  of  Lyapunov 
functions,  is  the  use  of  Hamiltonian  functions.  This  has  the  advantage  of  interpreting  the  time 
derivative  of  the  Hamiltonian  of  the  conservative  part  of  the  system  as  describing  the  power  transfer 
from  the  conservative  part  to  the  dissipative  part  of  the  system  [17].  .An  application  of  a  nonlinear 
version  of  MRAC  to  a  two  DOF  robot  arm  modelled  by  Hamiltonian  dynamics  using  a  single  rigid 
link  reference  model  led  to  a  nonlinear-discontinuous  control  law  [45].  The  Cartesian  space  control 
and  Artificial  Impedance  approach  for  the  Hamiltonian  systems  were  not  formulated  in  [45].  An 
interesting  alternative  to  the  joint  torque  computation  using  the  inverse  dynamics  (contained  in  the 
nonlinear  state  feedback  compensator)  is  the  use  of  a  disturbance  observer  for  estimating  the  same 
torque,  but  using  measurements  of  the  actuator  currents  and  joint  accelerations  (or  computations  of 
the  derivatives  of  joint  speeds)  [43],  This  results  in  a  suitable  joint  decoupling  if  sensors  bandwidths 
are  sufficient  and  the  computation  time  for  the  disturbance  observer  is  sufficiently  short.  Later 
formulation  of  the  Cartesian  control  of  a  joint  decoupled  robot  arm  was  based  on  a  resolved 
acceleration  control  scheme  and  can  also  be  used  for  the  Artificial  Impedance  approach  [44], 


4,0  TOE  CONCEPT  OF  IMPEDANCE  CONTROL 
4.1  The  Basic  Model 


37.  The  Artificial  Impedance  approach  can  be  conceptually  illustrated  for  a  simple  position 
control  problem.  We  assume  a  mass  m,  (positioned  at  x)  separated  from  a  moving  target 
(positioned  at  xj  by  an  artificial  elastic  spring  with  stiffness  K  in  parallel  with  an  artificial  damper 
with  coefficient  B.  A  term 
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Ml  where  M  is  a  virtual  mass,  is  also  included. 

The  equation  of  contact  motion  of  the  mass  is  given  by 

f  =  mX  +  (1) 

where  f^  is  the  contact  force. 

An  artificial  impedance  between  x  and  xd  would  be  described  in  this  case  by 


-  fm  -  M  X  +B(x-xd)  +  K(x-xd) 


(2) 


38.  The  inclusion  of  a  term  proportional  to  xd  in  this  equation,  as  in  [11,20],  requires 
assumptions  concerning  the  target  inertial  properties  (unit  mass  in  [20]  or  virtual  mass  M  in  [11]). 
In  this  paper  we  chose  an  artificial  impedance  structure  which  can  be  physically  reproduced  by  adding 
a  M-m  mass  to  the  mass  m  and  a  parallel  link  of  a  spring  K  and  damper  B  between  the  mass 
and  a  massless  target. 

39.  There  are  two  situations  considered  here  in  the  design  of  the  impedance  control  scheme: 
free  motion  of  the  mass,  and  contact  with  an  object.  In  the  case  of  free  motion,  only  x(t)  and  x(t) 
may  be  measured. 


4.2  Free  Motion  Control  of  a  Frictionless  Rigid  Body 


In  the  case  of  free  motion 


=  o 

(3) 

which  transform  (1)  and  (2)  into 

f  -  mx 

(4) 

and 

MX  +  B(x-xd)  +  K(x-.\j)  =  0 

(5) 

Eq.  (5)  gives  the  computed  acceleration 

x  (c)  =  M'1  B(V*)  +  M'1  K(xd-x) 


(6) 
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Substituting  x  in  eq.  (4)  by  x(c)  we  obtain 


f  =  +  M^KCvx)] 


(7) 


40.  The  control  law  (7)  and  the  system  equation  (4)  are  represented  in  Figure  2.  The  desired 
artificial  impedance  in  eq.  (5)  contains  three  parameters,  M,  B  and  K,  but  in  the  case  of  free  motion, 
only  two  controller  parameters,  M‘lB  and  M'*K,  can  be  independently  chosen  for  a  required  error 
dynamics  in  eq.  (7). 

41.  The  control  scheme  of  Figure  2  is  identical  to  a  PD  position  control  scheme.  The  artificial 
impedance  approach  provides,  however,  physically  meaningful  interpretation  of  the  controller  gains 
of  eq.  (7). 


4-3  Contact  Motion  of  A  Frictionless  Rigid  Body. 

42.  We  assume  that  the  motion  of  the  mass  m  is  impeded  by  an  obstacle  continuously  in 
contact  with  the  mass:  consequently,  the  motion  of  the  obstacle  is  imposed  on  the  motion  of  the 
mass,  x(t)=\j(t)  where  xt(t)  is  the  coordinate  of  the  obstacle.  Further,  wc  assume  that  the  motion 
of  the  obstacle  is  faster  than  the  motion  of  the  target  so  that  eventual  interception  is  possible.  The 
contact  of  the  mass  with  the  obstacle  produces  a  force  f^  which  can  be  measured.  The  states  x(t) 
and  x(t)  arc  measured  also. 

We  obtain  from  Eq.  (2),  the  computed  acceleration 


x  (c)  =  M’lB(vx)  +  M  'KfXj-x)  -  M*»  fm 


(S) 


Substituting  x  in(l)by  x  (e)  wc  obtain 


f  =  +  M-lK(x4.x)J.M’1  f*}  +fwt 


(9) 
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Figure  2.  Artificial  Impedance  Approach  scheme  for  a  m  in  free  motion 
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Artificial  Impedance  Avouch  scheme  for  a  mass  m  ui  am  tact  motion 
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43.  The  control  law  (9)  permits  the  calculation  of  the  force  command  f  for  given 
measurements  of  x,  x  and  f^,  and  known  values  for  xd  and  xd.  The  block  diagram  in  Figure  3 
shows  a  modulation  of  the  active  force  f  applied  to  the  mass  m  which,  besiaes  the  linear  control 
low  output  x  (c)  contains  also  a  feed-back  term  of  the  measurement  of  the  contact  force,  which  is 
not  present  in  PD  position  control  law.  For  M=m,  the  coefficient  of  f^,  1-mM'^Q,  and  the  contact 
force  does  not  influence  position  control  command  any  more. 


5.0  AUTONOMOUS  TRAJECTORY  GENERATION  AND  COLLISION  AVOIDANCE 
FOR  SINGLE  ARM  ROBOTS  USING  IMPEDANCE  CONTROLLERS 

5.1  The  Model 


44.  Assuming  that  the  measurement  of  Cartesian  contact  forces  6x1  vector  F,^,  between  the 
end  point  of  a  6DOF  jointed  robot  and  the  environment  is  available,  the  dynamic  equation  in  terms 
of  6x1  vector  0  of  joint  angles  gives  the  following  6x1  vector  r  of  joint  torques 

r  =  M(©)0  +  V(0,0)  +  JT(0)Fm.  (10) 


45.  The  term  M(0)  is  the  configuration-dependent  matrix  of  inertias.  Here  V(0,0) 
includes  Coriolis,  centrifugal,  gravitational  and  frictional  terms  and  J(@)  is  the  robot  Jacobian 
matrix.  Specific  to  the  .Artificial  Impedance  approach  is  the  choice  of  a  desired  artificial  impedance 
between  the  end  point,  positioned  at  X(a  6x1  vector)  and  the  target  positioned  at  Xd  (a  6x1  vector), 
producing  the  following  desired  dynamics 

-Fwt  =  MX  +  B(X-Xd)  +  K(X-Xj)  (11) 

M.  B  and  K  are  6x6  diagonal  matrices  of  the  desired  artificial  impedance  in  Cartesian  space.  From 
robot  kinematics,  the  following  relationships  between  state  variables  in  joint  space  and  in  Cartesian 
endpoint  space  can  be  written  [14] 

X  =  KIN(0)  (12) 

X  =  J(0)0  (13) 

X  =  j(Q)0  +j(Q)0  (14) 


where  the  Jacobian  6x6  matrice  J(0)  relates  joint  velocities  0  to  base  frame  defined  endpoint 
velocities  X.  Using  Eqs.  (12)  to  (14)  in  Eq.  (11),  can  be  written  as  a  function  of  0.  0.  0 
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-Fqj  =  mj  (©jd  +  -  sp^mco)]  (is) 

Based  on  measured  0.©  and  F^,  eq.  (15)  gives  the  computed  acceleration 

©  (c)  =  -J‘1(©)M'IFCfl-J‘1(0)j(0)0+J'l(0)M*1B(Xd-J(0)©l 

-r  J1(0)M*1K{Xd  -  KIN(0)]  (16) 

Substituting  0  in  eq.  (10)  by  ©(c)  we  obtain 

“  =  -M(0)J'*(0)M'*Fex,-M(0)J'1(0)j(0)©-t-M(0)J'i(0)M‘1B(Xd-J(0)©] 

t  M(0)J’1(0)M‘lK(Xd-KIN(0)]  +V(0.0)+JT(0)Fexl .  (17) 

of 

r  =  M(0)J'l(0)(M’1K[Xj-KlN(0)]+M’iB(Xd-J(0)©j-j(©)0-M'1FsW} 

r  V(6-S)  +  Jt(@)F„,  (IS) 

Using  the  notations  Mx  for  the  Cartesian  mass  matrix  and  Vx  for  the  velocity  dependent 
and  gravitational  terms,  we  obtain  [14] 

Mx(0)  =  J*t(0)M(0).J‘1(0)  (19) 

and 

Vx(0,0)  =  J  T(0)[V(0,©)-M(0)J"1(0)j(0)©]  (20) 


Using  the  notations  (19)  and  (20)  in  (18)  we  obtain 
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r  =  Jt(0){Mx(0)  {M'iK[Xd-KL\T(0)]-rM'iB(Xd-J(©)0]-M  IFca} 

-  (21) 

For  M=MJ(0),  the  coefficient  of  is  I-MLM'1=Q  and  does  not  influence  the  position  control 
commands. 

We  can  denote  the  computed  Cartesian  acceleration  by 

X  (c)  =  M'1K[Xd-KIN(0)]+M‘IB[Xd-J(0)0]-M‘1Fea  (22) 

Using  the  relationship  between  the  Cartesian  force  6x1  vector  F  and  r  [14],  F=J'i(0)r,we 
obtain  the  Cartesian  space  dynamics  equation 

F  =  M*(0)  X  (c)+Vx(0.0)-rFon  (23) 


For  F  and  X  (c)  defined  with  regard  to  the  robot  base 
frame,  Mx(0)  results  a  diagonal  matrix. 

46.  Equation  (2?.’)  gives  the  output  of  a  linear  Cartesian  controller  and  equation  (23)  contains 
the  Cartesian  decoupling  scheme  terms  Mx(0)X  (c),  Vx(0,0)  and  F^,  (see  Figure  4).  The  Cartesian 
decoupling  scheme  has  not  been  in  this  derivation  assumed,  but  rather  is  a  result  of  imposing  the 
desired  artificial  impedance  in  equation  (11)  between  the  robot  endpoint  and  the  target  as  a 
reference  model. 

47.  Collision  avoidance  features  can  be  incorporated  in  this  scheme  by  posing  artificial 
repulsive  impedances  between  the  robot  arm  and  the  obstacles  [11]. 


5.2  Difficulties  Associated  With  The  Application  Of  The  Artificial  Impedance  Approach 

To  Robot  Trajectory  Generation. 

48.  The  resultant  of  the  action  of  the  attractive  and  repulsive  impedances  is  zero  for  x=xd 
whenever  the  artificial  impedances  are  defined  to  simulate  the  effect  of  physical  impedances  applied 
to  the  same  Cartesian  decoupled  robot  arm  [9].  The  repulsive  artificial  impedances  were  previously 
reduced  to  virtual  springs  [9,20],  This  choice  results  in  a  large  reflection  on  the  resulting  repulsive 
potential  field  when  a  simple  obstacle  is  encountered.  Multiple  obstacles,  in  particular  configurations, 
c3n  lead  to  poorly  damped  or  unstable  robot  arms  motion  as  a  result  of  continuous  reflections  on  the 
resulting  artificial  repulsive  potential  fields  surrounding  the  obstacles.  This  is  also  shown  in  the 
simulation  results  described  in  this  paper.  The  choice  of  spring-damper  type  of  artificial  repulsive 


! 
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Figure  4.  Artificial  Impedance  Approach  in  a  Two  part  Control  Scheme 


impedances  creates  the  necessary  damping  associated  with  each  artificial  impedance.  This  solution 
produces  the  same  performance  for  multiple  obstacle  case  when  compared  to  the  solution  of  adding 
a  speed  dependent  term  added  to  a  conservative  artificial  impedance  or  potential  field,  as  in  [2,3]. 
only  if  the  velocity  dependent  term  is  applied  along  the  resultant  force  produced  by  the  combined 
effect  of  all  conservative  terms.  '  ' 

49.  Another  important  difficulty  in  this  case  is  the  possibility  of  local  static  equilibrium  points 
in  the  robot  Cartesian  space.  This  possibility  is  obvious  in  case  of  obstacles  which  create  potential 
field  concavities  in  the  path  of  the  end  effector  toward  the  desired  position,  i.e.  when  the  resultant 
of  the  repulsive  and  attractive  potential  fields  is  zero  for  X*Xd.  In  this  paper  artificial  impedance 
approach  is  used  for  trajectory  generation  and  correction  in  case  of  obstacles  by  paring  artificial 
impedances  in  the  Cartesian  space.  Further  modifications  of  the  choice  of  Cartesian  space  artificial 
impedances  can  lead  to  solutions  for  escaping  the  locai  potential  traps  by  using  a  coastal/surface 
navigation  scheme.  The  schemes  consist  of  Cartesian  tracking  of  virtual  attractive  points  moving 
along  a  safe  path.  In  the  case  of  a  pianar  manipulator,  coastal  navigation  scheme  consists  in  creating 
a  virtual  attractive  point  X,  which  will  move  continuously  on  a  path  orthogonal  to  the  vector  starting 
at  end  effector  position  and  ending  in  the  point  X0  detected  by  proximity  sensors  on  the  obstacle 
as  the  closest  to  the  end  effector.  As  long  as  the  line  of  sight  between  the  end-effector  position 

X  and  the  desired  position  Xd  is  obstructed,  the  desired  position  is  replaced  by  a  virtual  attractive 
point  X,  and  the  end  effector  will  move  toward  The  resulting  perpendicularity  of  the  vectors 
XX,  and  XX0  leads  to  a  trajectory  generation  on  a  path  along  the  coast  at  a  safe  distance  from  the 
obstacle.  Further  heuristics  can  be  used  in  both  planar  and  spatiai  motion  in  order  to  find  a  feasible 
path.  In  spatial  motion,  the  surface  navigation  scheme  defines  the  virtual  attractive  point  in  a  plane 
defined  by  X,  Xd  and  X0.  These  schemes  can  be  further  improved  based  on  minimum  energy 
requirements. 

50.  Another  difficulty  analyzed  in  this  paper  refers  to  a  general  problem  of  Cartesian-based 
controllers.  Given  the  particular  situation  in  which  the  actuators  and  the  sensors  are  located  at  joints 
and  the  errors  are  controlled  in  Cartesian  space,  the  verification  of  nonviolating  the  actuators’ 
saturation  limits  is  not  straightforward.  In  the  case  of  the  artificial  impedance  approach  applied  to 
a  Cartesian  decoupled  robot  a  solution  to  this  difficulty  is  a  linear  rescaling  of  the  Cartesian  force 
F  defined  by  equation  (23)  using  the  instantaneous  linear  dependence 


| 


F  -  JTt(0,)t 


(24) 


In  case  the  resulting  torque  command,  say  Tj,  violates  one  of  the  joint  actuation  saturation 
limit  ri  max,  i.e.  Tj  >  rj  max,  the  linear  rescaling  results  in  a  feasible  torque  command  rf  defined  by 


'  i, max 


)F  -  J'T(&)( 


1  i,max 


=  I-T* 


)T  =  J 


(25) 
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.  -  | 

i 

where  ~umxjT,  is  a  scalar  recalculated  at  every  computation  step.  This  scheme  is  not  applied  when  j 

all  joint  torques  are  within  the  saturation  limits,  r-t  <  r;  m,v.  The  result  is  a  set  of  commands  for  a  -  | 

slower  but  feasible  motion.  A  computationally  efficient  solution  has  been  proposed  for  a  joint  space  | 

dynamics  formulation  [46].  * 


53  Simulations  of  a  2  DOF  Manipulator  Using  Impedance  Control  of  the  Free  Motion 

51.  The  simulations  are  for  a  2  DQF  jointed  manipulator.  The  desired  impedance  parameters 
are 


M  =  I,, 
B  =  41,, 
K  =  4L, 


where  I,  is  the  2x2  identity  matrix.  The  manipulator  is  frictionless  and  operates  in  a  vertical  plane  gt 

with  gravitational  forces  included. 

The  inertia  matrix  is  defined  using  the  conventions  given  in  [14] 


M(©)= 


(?2-m,+2c,  i,m,cos(02) +512(m1+  m,)  J,2m, + ?2m2cos(0,) 
{22m,+?-?i{,m,  cos(0,)  0,2m, 


The  vector  of  velocity  dependent  and  gravitational  terms  is 


V(Q,©)  = 


-m,iit,  sinf©,)©,2  -  2m,51{,  sin(02)0102 
/  m,?^,  sin(@2)012 


m,J,g  cos(01+0,)+(m1+m:;){1g  005(0!) 


+ 


m,?,g  cos(0j  40,) 
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52.  In  these  equations  nt;,  6-t  and  0;  (for  i=l,2)  represent  the  mass,  length  and  joint  angle  of 
the  appropriate  Sink.  The  matrices  M^©)  and  Vx(@,0)  are  determined  using  Eqs.  (19)  and  (20). 

53.  The  paths  from  an  initial  position  I  to  a  destination  D  for  the  three  control  schemes 
are  compared  in  Figure  5.  Path  (a)  resuits  from  the  PD  position  controller  with  ^=£,=4.  The 
path  is  constrained  to  start  at  I  and  end  at  D,  but  is  otherwise  established  arbitrarily  by  the 
controller.  This  scheme  requires  an  inverse  kinematic  computation.  Path  (b)  is  the  result  for  the 
case  of  a  joint  decoupling  scheme.  The  result  indicates  that  the  performance  of  the  artificial 
impedance  based  controller  applied  to  a  joint  decoupled-linearized  manipulator  is  poor;  however, 
good  performance  of  the  artificial  impedance  approach  is  illustrated  by  path  (c)  which  results  for  an 
impedance  controlled  manipulator  in  which  a  Cartesian  decoupling  scheme  is  used.  The  straight  line 
path  is  obtained  without  an  inverse  kinematics  computation  or  a  trajectory  planning  scheme. 

54.  Several  simulations  were  run  to  illustrate  the  obstacle  avoidance  scheme.  The  results  for 
single  obstacle  avoidance  are  presented  in  Figure  6.  In  Figure  6a  the  positions  of  I  and  D  are  the 
same  as  for  the  simulation  which  resulted  in  a  straight  path  in  Figure  5c.  An  obstacle  0  is  on  this 
straight  line  between  I  and  D  and  the  impedance  based  controller  corrects  the  trajectory  to  avoid 
collision.  In  both  Figs.  6a  and  6b  the  avoidance  path  looks  like  a  reflection  on  an  imaginary  wall  that 
protects  the  obstacle  0.  After  avoiding  the  collision,  the  path  returns  (in  Figure  6a)  to  the  vicinity 
of  the  straight  line  path  of  Figure  5c. 

55.  Multiple  obstacles  avoidance  is  illustrated  in  Figure  7.  Figures  7a  and  7b  represent 
simulations  of  the  same  situation  except  for  the  repulsive  stiffness  which  is  two  times  higher  in  Figure 
6b  (Ko=200)  versus  Figure  7a  (K=100).  In  Figure  7a  the  path  goes  between  the  two  obstacles. 
Higher  repulsive  stiffness  diverts  the  path  around  both  0?  and  0{  as  shown  in  Figure  7b.  .Also  shown 
in  Figure  7a  is  the  straight  line  path  between  I  and  D,  which  is  obtained  when  no  obstacles  exist. 

56.  In  Figs.  7c  and  7d  the  same  data  are  used  except  for  a  five  times  higher  stiffness  in  the 
case  of  Figure  6d  (Ko=500)  versus  Figure  6c  (1^=100).  The  path  in  Figure  7c  results  from  a  first 
correction  from  the  repulsive  force  associated  with  Oj  followed  by  that  produced  by  the  repulsive 
force  associated  with  and  so  on  until  it  returns  toward  a  path  leading  towards  D.  In  Figure  7d  the 
reflections  between  0t  and  (h  are  much  stronger,  actually  preventing  the  manipulator  from  proceeding 
past  the  objects  to  D.  The  repulsive  force  is  produced  here  by  a  nonlinear  spring  and  the  motion 
is  unstable.  This  result  suggests  that  repulsive  forces  used  in  collision  avoidance  should  contain  a 
damping  term  to  avoid  instability  in  some  cases.  The  case  of  four  point-obstacles  is  illustrated  in 
Figure  7c.  The  value  of  the  chosen  repulsive  impedance  (1^=100)  leads  to  a  smooth  path  which 
reaches  D.  These  simulations  show,  in  general,  that  the  choice  of  the  structure  and  the  gains  used 
for  repulsive  forces  are  important  and  require  a  design  technique. 

57.  The  simulations  illustrated  in  Figures  5  to  7  assume  that  there  is  always  enough  actuator 
torque  tc  produce  the  command  torque  required  by  the  controller.  Separate  simulations  were 
performed  in  which  a  Cartesian  rescaling  scheme  based  on  Eq.  (25)  is  used  to. correct  the  command 
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torque  when  the  actuator  limits  are  reached  (Figs,  lib  and  12b).  For  comparison,  the  simulations 
in  Figure  8a  and  8a  show  the  effect  of  reaching  actuator  limits  when  no  correction  of  the  command 
torque  is  performed.  As  seen  in  Figure  8a,  the  path  in  this  latter  case  departs  from  the  straight  line 
1-0. 


58.  The  interception  of  moving  targets  is  illustrated  in  Figure  10.  In  Figure  10a,  a  target  is 
moving  in  a  straight  line  starting  at  Dj  and  is  intercepted  by  the  manipulator  at  Dj.  A  target  moving 
in  a  circular  trajectory  with  an  angular  speed  0.3  rad/sec  is  intercepted  rapidly  (Figure  10b),  while  if 
the  angular  speed  is  increased  to  3.0  rad/sec  the  target  is  never  intercepted.  The  artificial  impedance 
parameters,  M=l,  B=4  and  K=4,  correspond  here  to  a  natural  frequency  of  2  rad/sec. 


6.0  AUTONOMOUS  TRAJECTORY  GENERATION  FOR  DUAL  ARM  ROBOTS 
USING  IMPEDANCE  CONTROLLERS 

6.1  The  Model 


59.  In  this  paper  a  dual  arm  robot  handling  a  flexible  rod  and  a  string  is  assumed.  The  model 

of  the  dynamics  of  each  arm  is  an  extension  of  the  model  developed  by  Luh  and  Zheng  [21]. 

The  generalized  vector  of  input  forces  to  the  joints  of  the  n  DOF  leader  and  the  n  DOF 
follower  arms  are 


1 1-[Jb(©1)]TF1-[Ja(®1)]TNl-D(©1)<51-f(^>,.©l)-g(©i)  =0  (26) 

Tf-Jb((Of)TFf-[Ja(0f]TNf-D(0‘)0  f-f(0f,0f)-g(@f)=O  (27) 

where 

J(0)  =[.It,T(©).  J,W 

J(0)  =  the  Jacobian  matrix 

0  =  n-dimensional  vector  of  joint  angular  displacements 

F,N  =  the  vector  forces  and  moments,  respectively,  applied 
at  the  origin  of  the  end  effector  coordinates 

D(0)  =  n  by  n  inertia  matrix 

f(0,0)  =  n-dimensional  vector  of  Coriolis  and  centrifugal  terms 


Figure  S.  End-Effector  Path  for  the  Case  of  Actuator  Torque  limits  study 

(a)  No  Corrective  Action 

(b)  With  Rescaiing  of  the  Cartesian  Force  Vector 


(Nn)  T2  (Nm) 


Figure  9. 


Joint  Torques  for  the  Cases  (a)  and  (b)  in  Figure  8 


Rate  0.3 
Rate  3.0 
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g(0)  .  =  n-dimensional  vector  of  gravity"  terms  and, 

C,f  =  superscripts  for  the  leader  and  the  follower  respectively. 

60. The  vectors  F  and  N  result  from  specific  load  dynamics  model.  The  leader  and  follower 
end  effectors  are  assumed  linked  by  artificial  mechanical  impedances  to  the  target  position.  The 
control  scheme  is  similar  to  the  control  scheme  for  single  arms,  given  in  Figure  4. 

61.  A  dual-arm  planar  horizontal  robot  with  two  degrees-of-freedom  arms  is  simulated  (Figure 
11)  [31].  The  handled  objet is  assumed  massless,  with  an  unstressed  length  of  0.2  m.  The 
parameters  of  the  two  arms  are  L^O^m,  1^-0  A  m,  m^O.4  kg,  m2’=0.4  kg,  Lf=0.5  m,  l^-QA  re, 
m1c=0.4  kg.  m2f=0.4  kg,  d=1.0  m.  The  artificial  impedance  parameters  are  M=1.0  kg,  B=4.0  Ns/m 
3nd  K=4.0N/m. 


6.2  Simulations  of  a  Robot  Handling  a  Flexible  Rod 

62.  The  flexible  rod  is  modelled  as  a  linear  spring  with  a  damper  in  parallel,  having  a  spring 
constant  of  1.0  N/m  and  a  damping  constant  of  1.0  Ns/m.  The  end  effectors  are  located  initially  at 
R,1  and  R/,  respectively,  and  move  in  the  approaching  phase  to  Rj1  and  where  the  flexible 
rod  is  grasped  (Figure  11).  In  the  next  phase  the  flexible  rod  is  transported  to  Rd2fand  Rd2f  where 
the  leader  reaches  the  final  position.  Subsequently,  only  the  follower  moves,  rotating  the  rod  until 
the  follower  reaches  its  final  position  in  Rd3f.  The  length  of  the  flexible  rod  varies  in  time  within 
±  1%.  Further  simulations  in  which  the  sampling  time  is  reduced  from  0.005  s  to  0.001  s  leads  to 
a  reduction  of  the  length  variation  by  a  factor  of  50.  These  simulations  show  that  the  impedance- 
based  controller  can  coordinate  the  motion  of  a  two-arm  robot  in  a  sequential  fashion.  Improvements 
to  the  control  scheme  discussed  in  the  next  section, eliminate  this  sequential  approach  and  result  in 
a  smooth  simultaneous  motion  of  the  two  arms  to  the  final  positions.  This  control  scheme  is  tested 
in  handling  a  string. 


63  Simulations  of  a  Robot  Handling  a  String 

63.  The  string  is  modelled,  when  stretched,  as  a  very  compliant  spring  and,  when  compressed, 
as  producing  no  reactive  forces  on  the  end  effectors.  Only  the  phase  in  which  the  object  is  actually 
handled  is  simulated  here  since  the  approaching  phase  poses  the  same  coordination  problems  for  all 
tasks.  The  initial  positions  of  the  end  effectors  are  Rj1  and  R/,  respectively,  where  the  object  is 
assumed  grasped  (Figure  12).  The  artificial  impedances  created  between  the  initial  positions  of  the 
end  effectors  and  their  desired  positions  Rd  and  Rdf,  respectively,  generate  straight  line  trajectories 
in  the  case  of  a  string  which  when  compressed  does  not  produce  any  reactive  forces  (Figure  13).  The 
distance  between  the  two  end  effectors  varies  during  the  motion  with  a  minimum  value  of  0.14  m 
(Figure  14),  which  leads  to  a  large  sag  in  the  string  of  0.2  m.  An  artificial  impedance  approach 
overcomes  the  problem  of  the  lack  of  reactive  forces  when  compressing  a  string  in  length  by  creating 
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an  artificial  impedance  in  parallel  with  the  string.  Simulations  were  performed  for  an  artificial  parallel 
impedance  with  a  spring  constant  of  1.0  N/m  and  a  damper  constant  of  2.0  Ns/m.  Simulations  results 
in  Figure  15  show  that  the  leader’s  end  effector  keeps  the  straight  line  trajectory  while  the  follower’s 
end  effector  trajectory  is  generated  such  that  the  string  during  the  motion  has  practically  no  sag,  as 
shown  on  Figure  16. 


7.0  IMPEDANCE  CONTROL  OF  MOBILE  ROBOTS 

64.  In  the  Iasi  few  years  impedance  control  and  artificial  impedance  approach  were  used  for 
mobile  robot  motion  control  and  collision  avoidance  [48]  to  [50].  The  results  were  presented  for  a 
simplified  case  which  the  dynamics  of  the  mobile  robot  body  can  be  neglected  and  direct  speed 
control  is  possible.  Full  dynamic  models  for  mobile  robots  were  reported  recently  [51,52].  We  are 
currently  developing  an  impedance  controller  based  on  a  complete  robot  dynamics  model 

8.0  IMPEDANCE  CONTROL  IMPLEMENTATION  ISSUES 

1 

65.  In  order  to  achieve  the  target  impedance  described  by  Eq.  (11)  in  Cartesian  space,  the 
overall  system  dynamics  of  the  robot  arm-controller-actuator  have  to  be  reduced  to  independent 
second  order  linear  passive  systems.  A  6  DOF  robot  arm  would  behave,  then,  as  6  independent 
passive  inertial  systems  in  Cartesian  space. 

8.1  Nonlinear  State  Feedback  Compensation  j 

66.  The  implementation  of  the  .Artificial  Impedance  approach  was  initially  based  on  using  a 
two-part  control  [34]  or  a  partitioned  control  law  [14]  which  contains  a  linear  servo  control  law  in 
Cartesian  space  and  a  nonlinear  state  feedback  compensator.  In  Figure  4,  Eqs.  (22)  and  (23)  are 
represented  in  a  diagram  in  which  the  linear  servo  control  output  X  is  applied  to  a  nonlinear 
state  feedback  compensator  which  consists  of  Mx(0)X(c)  +  Vx(@,©)  +  F^.  For  F,^  =  0  this  type 
of  Cartesian  space  control  has  been  known  earlier  as  Resolved-Acceleration  Control  [17]. 

67.  The  decoupling  method  for  nonlinear  systems  has  been  formulated  by  Freund  and  has 

been  illustrated  for  a  joint  decoupling  case  [20].  An  interesting  implementation  of  the  two-part  r 

control  scheme  consists  of  generating  the  joint  trajectory  by  tracking  a  "false  target"  point  in  the 
phase  space  of  each  de-  coupled  joint  [32].  Cartesian  tracking  of  virtual  attractive  points,  described 
in  Chapter  III,  can  be  seen  as  a  Cartesian  translation  of  the  "false  target"  method  for  joint  space. 

Generally,  joint  space  decoupling  has  been  more  extensively  analyzed  than  Cartesian  space  decoupling 
[34,35,36].  .  "  r,  . 

68.  The  artificial  Impedance  approach  and  its  continuum  mechanics  counterpart  the  artificial  •  ‘ 

Potential  Field  approach,  are  special  cases  of  Cartesian  control  in  which,  not  oniy  the  errors  are 

controlled  in  Cartesian  space,  but  the  robot  arm  is  reduced  for  the  linear  controller  to  a  decoupled 
Cartesian  system  of  independent  inertial  systems.  Khatib  formulated  the  problem  in  the  operational 


Figure  11.  Dual-arm  planner  robot 
used  for  simulations 
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Figure  13.  Simulation  of  a  robot  handling  a  string 


Figure  14.  Time  variation  of  a  the  distance  between 
die  two  end  effectors  for  the  case  simulated  in  Fig.  i3. 


Figure  15.  Simulation  of  a  robot  handling  a  string 
when  a  artificial  impedance  is  created  in  parallel 
to  the  string 


Figure  16.  Time  variation  of  the  distance  between 
the  two-effectors  for  the  case  simulated  in  figure  15. 
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space  of  the  robot  end-effector  [1,19].  Given  a  variety  of  terms  for  similar  concepts  (task  space,  end- 
effector  space,  hand  space,  operational  space)  which  ail  refer  to  a  Cartesian  space  we  have  used  in 
this  paper  only  Cartesian  space  terms.  Khatib  resuits  are  similar  to  those  incorporated  in  Figure  3, 
i.e.  a  Cartesian  two-part  control  scheme,  except  they  were  developed  originally  in  continuum 
mechanics  (Artificial  Potential  Field)  formulation.  Hogan  et  al  developed  a  further  Artificial 
Potential  Field  approach  and  an  Artificial  Impedance  approach  using  also  a  Cartesian  two-part 
control  law  [3,7,8].  An  inverse  kinematics  scheme  for  Cartesian  space  control  has  been  proposed 
without  the  coupling  considerations  [33].  The  implementation  issues  of  a  Cartesian  two-part  control 
law  refer  here  mainly  to  difficulties  in  achieving  the  Cartesian  decoupling  of  a  highly  nonlinear  robot 
arm.  The  parameters  of  the  dynamic  equations,  for  example  the  load  parameters,  are  often 
imprecisely  known  and  an  analysis  of  the  uncertainty  effect  on  the  Cartesian  controlled  robot  arm 
motion  is  needed.  The  error  dynamics  equation  in  this  case  is  still  describing  a  coupled  nonlinear 
system  as  a  result  of  uncertainty  of  the  parameters  of  the  dynamic  model  used  for  non.inear 
compensation  [36].  This  situation  led  to  various  solutions  proposed  for  improving  robustness  of  the 
Cartesian  control  law,  in  general,  and  of  the  Artificial  Impedance  approach,  in  particular. 

$.2  Robustness  Issues 


69.  Several  solutions  were  proposed  for  robust  robot  control.  Some  solutions  are  applicable 
to  the  Artificial  Impedance  approach  as  for  example  Model  Reference  Adaptive  Control  [37,38,39] 
and  Second  Method  of  Lyapunov-based  designs.  Model  Reference  Adaptive  Control  can  be 
formulated  for  a  reference  model  which  represents  a  set  of  three  independent  translational 
mechanical  impedances  and  three  independent  rotational  mechanical  impedances,  i.e.  six  second  order 
linear,  constant  parameter  differential  equations.  The  error  between  the  reference  model  positions 
and  Cartesian  end-effector  positions  would  be  used,  in  this  case,  for  the  adaptation  mechanism.  This 
approach  benefits  from  the  robustness  of  the  MRAC  approach,  but  only  asymptotically  the  robot  arm 
would  behave  like  the  desired  Cartesian  space  impedances. 

70.  Second  Method  of  Lvapunov-based  designs  have  been  used  for  obtaining  sliding  mode 
controllers  for  single  [41]  and  dual  arm  robots  [23].  The  application  of  a  sliding  controller  for 
implementing  an  Artificial  Impedance  approach  has  been  explicitly  formulated  by  J J.  Slotine  [41,  pp. 
214-215].  In  this  case,  sliding  surfaces  are  chosen  as  linear  functions  of  the  errors,  for  the  Cartesian 
state  variables,  and  of  the  environment  contact  forces.  In  sliding  mode,  the  controller  would  make 
the  robot  behave  like  the  desired  impedance.  The  resulting  discontinuous  control  law,  inherent  to 
the  application  of  the  second  method  of  Lyapunov,  can  be  approximated  by  a  boundary  layer-based 
continuous  control  law.  Maintaining  sliding  mode  requires  sufficient  torque  output,  large  actuator 
bandwidths  and  fast  closed  loop  components  other  than  the  robot  arm. 

71.  Second  Method  of  Lyapunov  can  be  applied  directly  for  obtaining  a  corrective  term  added 
to  a  linear  control  law  in  order  to  compensate  for  the  errors  resulting  from  inexact  cancellation  of 
the  nonlinear  dynamic  terms  by  nonlinear  state  feedback  compensator.  Again,  a  discontinuous 
corrective  term  would  result  and  a  continuous  approximation  solution  is  proposed  for  joint  decoupling 
case  [18,  pp.  227-236],  A  similar  formulation  can  be  developed  for  the  Cartesian  decoupling  case  and 
the  Artificial  Impedance  approach  case.  An  interesting  alternative  in  this  case,  to  the  use  of 


Lyapunov  functions,  is  the  use  of  Hamiltonian  functions.  This  has  the  advantage  of  interpreting  the 
time  derivative  of  the  Hamiltonian  of  the  conservative  part  of  the  system  as  describing  the  power 
transfer  from  the  conservative  part  to  the  dissipative  part  of  the  system  [16].  A.i  application  of  a 
nonlinear  version  of  MRAC  to  a  two  DOF  robot  arm  modelled  by  Hamiltonian  dynamics  using  a 
single  rigid  link  reference  model  led  to  a  nonlinear  discontinuous  control  law  [44].  The  Cartesian 
space  control  and  the  Artificial  Impedance  approach  for  the  Hamiltonian  systems  were  not 
formulated  in  [44].  An  interesting  alternative  to  the  joint  torque  computation  using  the  inverse 
dynamics  (contained  in  the  nonlinear  state  feedback  compensator)  is  the  use  of  a  disturbance 
observer  for  estimating  the  same  torque,  but  using  measurements  of  the  actuator  currents  and  joint 
accelerations  (or  computations  of  the  derivatives  of  joint  speeds)[42].  This  results  in  a  suitable  joint 
decoupling  if  sensors  bandwidths  are  sufficient  and  the  computation  time  for  the  disturbance  observer 
is  sufficiently  short.  Later  formulation  of  the  Cartesian  control  of  a  joint  decoupled  robot  arm  was 
based  on  resolved  acceleration  control  scheme  and  can  also  be  used  for  Artificial  Impedance 
approach  [43]. 


72.  The  implementation  of  a  Cartesian  two-part  control  law  represented  in  Figure  4  requires 
extensive  computations  of  the  highly  nonlinear  terms  MX(Q)  and  Vx(©,0)  besides  the  kinematics 
computations  present  in  ell  Cartesian  control  schemes.  Compared  to  Cartesian  decoupling  schemes, 
joint  decoupling  schemes  are  computationally  less  intensive.  The  proposed  disturbance  observer 
scheme  proposed  in  [43]  is  an  example  of  computationally  efficient  joint  decoupling  scheme. 


In  what  follows  is  a  proposed  scheme  based  on  a  three-part  control  scheme  which  can  be 
obtained  from  F.qs.  (18)  and  (22) 


<“>;(c>  =  J'HOHX  ^>-j(0)0} 


r  =  M(0)€>  <c)  +  V(0,0)  +  JT(0)Fat 


(29) 


73.  These  equations  are  represented  in  Figure  17  where  the  three  parts  of  the  control  scheme 
are  identified.  Eq.  (29)  corresponds  to  a  joint  decoupling  scheme,  while  Eq.  (28)  contains  the 
Cartesian  decoupling  scheme  of  a  joint  decoupled  manipulator.  As  remarked  earlier,  with  regard  to 
Eq.  (21),  for 

'  M  =  Mx(0)  =  (J-1)tM(0)J'1(©)  (30) 

74.  does  not  influence  any  more  the  position  control  commands  for  the  contact  motion. 
We  note  that  no  desired  contact  force  is  considered  in  this  scheme.  Condition  (30)  would  require, 
however,  a  desired  impedance  term  M  which  would  be  equal  to  the  configuration  dependent 
M,(0).  In  cases  when  the  desired  impedance  would  not  require  a  configuration  independent  mass 
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term,  the  computation  of  M,(0),  as  shown  in  Eq.  (30),  is  not  particularly  difficult  given  that  it  uses 
matrices  which  are  anyways  needed  for  decoupling  computations.  Moreover,  for  the  case  that  all 
elements  of  the  vector  X  are  defined  with  regard  to  the  inertial  frame,  Mx(0)  is  a  diagonal  matrix 
with  configuration  dependent  diagonal  elements.  It  can  be  observed  that  the  two-step  control  of 
Figure  3  has  been  replaced  in  Figure  17,  by  a  three-step  controk(a)  linear  Cartesian  control  law,  (b) 
Cartesian  decoupling  of  a  joint  decoupled  arm  and  (c)  joint  decoupling.  In  this  scheme,  step  (b) 
requires  only  manipulations  of  the  Jacobian  matrix  i.e.  only  kinematics  computations.  In  the  case 
of  no  contact  force,  Fext=0,  this  scheme  is  the  same  as  Figure  6  in  [17]  for  the  Resolved-Acceleration 
Control  for  which  the  PD  gains  can  be  interpreted  as  M'XK  and  M'*B.  Artificial  Impedance  approach 
brings  extra  compensation  for  the  contact  force  and  a  physical  interpretation  of  the  PD  gains.  In  fact, 
Resolved  Acceleration  control  did  not  produce  the  development  of  collision  avoidance  schemes 
probably  because  of  not  incorporating  the  Cartesian  control  of  actuators  in  an  overall  dynamic  model 
in  which  the  actuator  effects  are  replaced  by  passive  artificial  impedances  in  Cartesian  space. 

9.0  CONCLUSIONS  AND  RECOMMENDATION 

75.  Artificial  Impedance  approach  .is  an  important  candidate  for  manipulation  control  of 
complex  tasks.  Collision  avoidance,  actuator  limits  and  moving  target  interception  are  features  that 
can  be  naturally  implemented  in  an  impedance  controller  without  need  for  complete  geometric 
description  of  the  work  volume  and  of  the  intruding  obstacles. 

76.  Further  research  in  the  implementation  of  the  art'Ticial  impedance  approach  holds  the 
promise  of  an  autonomous  manipulator  which  generates  and  corrects  on-line  the  trajectory  when 
obstacles  or  actuator  limits  are  encountered. 

77.  The  coordinated  motion  of  a  dual-arm  robot  can  also  be  controlled  by  applying  the 
Artificial  Impedance  approach.  The  simulations  were  performed  to  illustrate  the  manipulation  of 
non-rigid  bodies  such  as  a  flexible  rod  and  a  string.  The  results  show  that  this  control  approach  is 
a  candidate  for  solving  some  of  the  difficulties  of  coordinated  dual-arm  robot  operation  both  in  the 
approaching  phase  and  in  the  object  handling  phase,  in  particular,  for  the  case  of  handling  a  string 
with  sag  constraints  and  other  non-rigid  objects.  Further  simulations  and  experiments  are  required 
at  this  stage  to  show  the  practicality  of  the  Artificial  Impedance  approach  to  dual-arm  trajectory 
generation.  The  implementation  of  the  Artificial  M-B-K  Impedance  approach  is  proposed  in  the 
form  of  a  three-part  control  scheme:  (a)  a  linear  Cartesian  control  law  with  position  gain  M_1K  and 
velocity  gain  M_1B,  (b)  a  Cartesian  decoupling  scheme  of  a  joint  decoupled  manipulator  using  only 
Jacobian  manipulations,  and  (c)  a  non-linear  feedback  joint  decoupling  compensator.  In  the  case 
of  non  contact  motion,  this  scheme  is  identical  to  Resolved  Acceleration  control.  The  importance 
of  an  Artificial  Impedance  control  scheme  results  from  the  physical  interpretation  of  the  linear 
controller  gains  and  the  possibility  of  generating  and  correcting  robot  trajectoiy  in  case,  of  detecting 
obstacles  by  posing  artificial  impedances  in  the  robot  workvoiume. 

78.  The  Impedance  Controller  as  proposed  in  this  paper  is  embedded  in  a  multipurpose 
hierarchial  control  structure.  Such  hierarchial  control  architecture  provides  a  means  to  introduce 
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higher  level  autonomous  operational  capabilities  which  are  of  critical  importance  to  many  military 
operations.  Supervised  Autonomous  Robots  and  eventually  Fully  Autonomous  Robots  will  employ 
hierarchial  control  architecture  which  basically  consists  of  three  modalities:  sensing,  action  and 
planning  with  their  interactions  within  a  uniform  structure.  Unfortunately,  Autonomous  Robotic 
Systems  are  still  at  their  infancies  due  to  various  reasons.  Some  of  them  are: 

■r  /'  , 

(i)  Manipulation  tasks  are  highly  unstable; 

(ii)  Sensory  feedback  is  often  inadequate; 

(iii)  Performamce  of  manipulators  is  limited; 

(iv)  Computational  complexity  is  intractable; 

(v)  Theoretical  foundation  do  not  yet  exit. (i.e.  nonlinear  system). 

79.  However,  current  research  activities  focus  on  the  development  of  robots  sensing,  action 
(execution)  such  as  trajectory  planning  and  object  avoidance  and  planning  and  an  overall  system 
concept  where  all  these  modalities  are  uniformly  integrated.  Military  operations  will  greatly  benefit 
from  these  developments.  ' 
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14.  Abstract: 

Robotic  Systems  Technology  is  under  contract  to  build  and  deliver  14  Surrogate 
Teleoperated  Vehicles  (STV)  to  the  DoD  Unmanned  Ground  Vehicle  Joint  Program  Office, 
The  STV  system  consists  of  two  parts  -  the  Remote  Platform  (RP)  and  the  Mobility/RSTA 
Module  (MOB/ RSTA).  The  RP  is  based  on  an  off-the-shelf  Polaris  six-wheel  drive, 
Ackcrman-stcered  all-terrain  vehicle.  Its  full  suspension  allows  the  operator  to  drive  at 
speeds  exceeding  58  kph,  The  current  automatic  drive  train  will  be  modified  using  a  unique 
dual  motor  hybrid  concept  that  incorporates  an  electric  motor  to  provide  slow  speed 
mobility  when  an  ultra-quiet  mode  is  required.  All  electronics  are  packaged  in  waterproof 
enclosures  that  allow  for  easy  changcout  of  electronic  components  for  simple  and  rapid 
maintenance  operations.  The  modular  MOB/RSTA  module  consists  of  an  elevating  mast 
and  a  pan  and  tilt  turret  with  a  variety  of  sensors,  including  a  laser  designator  and  a  FUR. 
The  mast  will  extend  3  meters  from  the  resting  position,  allowing  the  cameras  to  be  5 
meters  from  the  fround.  The  electronics  for  the  control  of  all  turret  functions,  including 
sensor  interfaces,  arc  totally  self-contained  within  the  turret. 
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SURROGATE  TELEOPERATED  VEHICLE  (STV)-  A  MODULAR  TESTBED 


1.0  INTRODUCTION 

Robotic  Systems  Technology  (RST)  is  under  contract  to  design,  build  and  deliver  14 
Surrogate  Teleoperated  Vehicles  (STV)  by  December,  1991  to  the  Department  of  Defense 
(DoD)  Unmanned  Ground  Vehicle  Joint  Program  Office.  This  office  is  located  at  the  U.S. 
Army  Missile  Command  in  Huntsville,  Alabama. 

The  STV  is  a  critical  step  in  the  acquisition  of  unmanned  ground  vehicles  for  combat 
and  combat-support  missions.  The  STV  will  provide  a  system  for  the  service  user  to  develop 
and  validate  his  employment  concepts  and  complete  Early  User  Test  and  Evaluation  to  aid  in 
the  development  of  requirements  for  future  unmanned  ground  vehicle  systems.  In  these  days 
of  declining  force  structure  and  the  increasing  emphasis  on  low-intensity  conflict  scenarios, 
these  systems  will  provide  the  Services  with  increased  force  multiplication  and  soldier 
survivability 

The  key  features  of  our  STV  design,  shown  in  Figure  1,  are  as  follows: 

•  The  use  of  a  proven,  commercially  available,  highly  capable  platform  to  minimize 
technical  and  schedule  risk  and  maximize  testing  availability. 

•  An  automatic  transmission  and  easy-to-operate  controls  to  minimize  the  time  required 
to  leam  how  to  operate  the  system. 

•  Six-wheel  drive  to  provide  maximum  all-terrain  mobility  and  excellent  stability. 

•  Ackerman  steering  and  full  suspension  chassis  design  to  permit  safe  high-speed 
driving  in  excess  of  58  kph. 

•  Rugged,  proven  chassis  frame  that  supports  up  to  682  kg  of  payload. 

•  Wide  wheelbase  and  low  center  of  gravity  to  allow  for  operation  on  a  35-degree 
slope. 

•  Increased  combat  survivability  due  to  the  small  physical  signature  of  the  platform  — 
1.3  meters  wide,  2.75  meters  long,  3  meters  tall. 

•  The  use  of  pneumatic  shocks  to  raise  and  lower  the  chassis,  thus  providing  the  ability 
to  create  a  rigid  suspension,  minimizing  vehicle  body  swaying  when  the  sensor  mast 
is  erected. 


Instantaneous  switching  between  direct  and  remote  driving  with  fully  backdrivable 
remote  actuation  components. 

The  capability  to  be  transported  in  the  bed  of  a  HMMWV  or  to  be  towed,  reducing 
logistics  requirements. 

Versatile  dual-engine  hybrid  design  with  a  25 -hp  diesel  engine  providing  high¬ 
speed/high-power  mobility  needs  and  a  DC  electric  drive  motor  allowing  for  an  ultra¬ 
quiet  mobility  mode. 
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Figure  1  The  Surrogate  Teleoperated  Vehicle  Concept 
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•  Fuel  capacity  of  40  liters  to  provide  a  driving  range  of  450  km. 

•  200  A-hr  of  batteries  that  provide  3  days  of  operation  in  the  limited  surv  :illance 
mode. 

•  Swimming  capability  using  dual  high-speed  propellors  for  water  mobility. 

•  Innovative  driver-down  position  for  lower  center  of  gravity,  true  roll-over  protection, 
and  the  capability  to  add  lightweight  armor  for  increased  protection. 

•  An  open  architecture  with  an  industry  standard  VMEbus  and  VxWorks  software 
development  system  to  facilitate  the  addition  of  increased  technology. 

•  A  modular  design  that  utilizes  the  concept  of  line  replaceable  units  to  minimize 
system  repair  time. 

•  Proven  elevated  mast  design  that  provides  3  meters  of  extension,  placing  the  sensor 
module  cameras  at  a  total  height  of  5  meters  feet  above  ground  level. 

»  A  MOB/RSTA  sensor  module  with  ±270  degrees  of  pan  motion  and  ±  90  degrees  of 
tilt  motion. 

•  Turret  interface  and  control  electronics  are  totally  self-contained  in  the  turret 
structure,  reducing  the  required  size  and  complexity  of  the  MOB/RSTA  interface  wire 
harness  from  150  to  25  wires. 

•  High-precision,  high-speed  pan  and  tilt  DC  servo  gearmotors  with  the  dynamic  range 
and  system  response  needed  to  support  accurate  target  tracking  and  the  use  of  head 
tracking  display  systems. 

•  A  modular,  “mix  and  match”  turret  design  that  allows  for  the  simultaneous  use  of  four 
different  day/night  stereo  driving  cameras  or  targeting  camera  systems  for  maximum 
mission  flexibility. 

•  A  differential  GPS  receiver  and  a  proven  dead  reckoning  approach  providing  accurate 
location  and  navigation  information. 

•  A  proven,  field-tested  passive  acoustic  detection  system  that  not  only  provides  true 
360-degree  binaural  hearing,  but  also  provides  automatic  target  cuing  to  reduce 
operator  workloads. 

The  STV  system  consist  of  four  parts  -  the  Remote  Platform(RP),  the  Mobility, 
Reconnaissance,  Surveillance,  and  Target  Acquisition  Module  (Mob/RSTA),  the  fiber  optic 
and  radio  frequency  Communication  System,  and  the  man-portable  Operator  Control  Unit. . 
This  current  paper  will  only  discuss  the  design  of  the  RP  and  the  Mob/RSTA  components. 

2.0  REMOTE  PLATFORM 

The  remote  platform  (RP)  system  consists  of  the  carrier  base,  the  and  the  remote  control 
electronics  that  provide  an  interface  to  all  systems  on  the  STV  and  the  communication 
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system.  Each  system  has  been  carefully  designed  to  work  as  an  integrated  system  with 
maximum  capability  and  minimum  risk  for  assembly  and  integration.  Each  subsystem  is 
described  individually  in  the  following  subsections. 

2.1  Carrier  Base 

The  Remote  Platform  carrier  base  is  a  modified  Polaris  Big  Boss  six-wheel,  six-wheel 
drive,  all-terrain  vehicle.  This  American  made,  commercially  available  platform  sells  for 
$4,899.  The  full  suspension  used  on  all  of  Polaris’  ATVs  allows  for  routine  safe  travel  to 
speeds  exceeding  58  kph.  The  use  of  six  wheel  drive  offers  a  50%  increase  in  mobility  over 
a  four-wheel  drive  vehicle  and  provides  reduced  tire  ground  pressure  when  compared  to  a 
four-wheeled  platform. 

With  a  wheelbase  of  1.9  meters, a  total  length  of  2.76  meters,  a  total  width  of  1.3 
meters,  and  a  total  weight  of  455  kg,  the  carrier  base  will  fit  in  the  rear  bed  of  a  HMMWV. 

In  addition,  the  carrier  base  is  fully  towable  to  allow  a  two-man  HMMWV  crew  to  transport 
more  than  one  vehicle  at  a  time. 

The  proven  frame,  suspension,  and  drivetrain  for  the  carrier  base  has  been  in  production 
for  3  years  on  the  Big  Boss  4X6.  The  are  no  changes  in  the  frame  between  the  4  X  6  and 
the  6X6;  only  the  drivetrain  changes.The  frame  will  be  slightly  modified  in  the  area  that 
mounts  the  existing  gasoline  engine  to  allow  for  mounting  of  the  larger  diesel  engine .  The 
frame  is  designed  to  easily  support  a  payload  of  682  kg.  The  operator’s  seat  is  moved 
forward  and  downward  to  lower  the  center  of  gravity  of  the  entire  system  and  to  give  the 
driver  roll  bar  protection.  This  will  also  allow  for  the  future  addition  of  lightweight 
composite  armor  to  protect  the  driver  from  small  arms  fire  or  top  armor  for  fragment 
protection. 

The  suspension  provides  for  maximum  mobility  in  rough  terrain  and  safe  high  speed 
travel  on  relatively  smooth  teirain.  The  existing  mounting  locations  for  the  suspension  will 
not  change.  The  front  two  wheels  use  a  MacPherson  strut  assembly.  Dual  shocks  are  used 
on  the  rear  in  a  trailing  arm  configuration.  The  standard  coiled  springs  will  be  replaced  with 
pneumatic  springs.  These  pneumatic  springs  allow  the  STV  system  to  raise  or  lower  by 
15.24  cm.  At  the  springs  bottom  position,  the  spring  effect  is  eliminated,  making  the 
platform  very  rigid,  with  the  tires  providing  the  only  suspension.  This  will  eliminate  any 
swaying  of  the  MOB/RSTA  module  due  to  motions  of  the  suspension,  a  common  problem 
when  mounting  a  mast  on  a  vehicle  with  a  suspension.  The  ability  to  vary  the  height  of  the 
vehicle  also  will  allow  the  operator  to  reduce  the  vehicle’s  visible  signature  if  needed  for 
stealth  or  transportation  purposes. 

The  main  motive  power  source  is  a  Fuji  25  hp,  997  cc,  diesel  engine.  This  three- 
cylinder,  water-cooled  engine  has  five  more  horsepower  than  the  commercial  Big  Boss  6X6 
two-stroke,  gasoline  engine.  This  engine  incorporates  the  latest  in  sound  quieting  technology 
that  makes  it  4  dB  quieter  than  any  other  known  diesel  engine  in  its  power  range.  To  further 
muffle  the  diesel  engine,  the  engine  and  drive  train  will  be  placed  in  a  sound-quieting  box  in 
the  middle  of  the  vehicle  behind  the  driver. 

In  addition,  the  exhaust  manifold  will  be  routed  through  the  long  suspension  tunnel  that 
runs  down  the  middle  of  the  back  part  of  the  vehicle.  This  long  sound  chamber  will  further 
dissipate  the  sound  levels  of  the  engine  exhaust  and  will  direct  the  exhaust  toward  the  ground 
to  reduce  the  transmission  distance  of  the  noise.  Also,  since  the  outside  walls  of  this  tunnel 
are  not  visible  from  the  side,  this  will  act  as  a  thermal  cooling  tunnel  (like  the  cooling  tunnels 
on  stealth  aircraft)  to  reduce  the  thermal  signature  of  the  platform  due  to  exhaust  emissions. 
Beneath  the  engine  compartment  are  two  20-liter  fuel  tanks  that  provide  a  total  operating 
range  of  450  km. 
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The  drive  train,  shown  in  Figure  2,  is  a  modification  of  the  current  6X6  drivetrain. 

The  rotational  speed  of  the  output  shaft  of  the  diesel  is  increased  from  3600  rpm  to  7200  rpm 
to  allow  Polaris  to  use  standard  6X6  drivetrain  components.  This  is  accomplished  by 
adding  a  2: 1-cog  belt  overdrive  to  the  output  shaft  of  the  diesel.  This  feeds  a  standard 
Polaris  Variable  Belt  Transmission  (PVT)  that  provides  power  to  the  gearbox.  This  PVT  is 
used  on  each  of  the  80,000  snowmobiles  and  ATVs  that  Polaris  produces  each  year. 

The  output  of  the  PVT  goes  into  a  gear  box  that  allows  changing  of  gears  into  either 
neutral,  reverse,  or  a  low  or  high  speed  range.  The  low  speed  range  provides  extra  pulling 
torque  for  steep  hill  climbing  or  vehicle  towing  but  limits  the  maximum  speed  to  around  33 
kph  (similar  to  the  low  gear  range  in  an  automobile  automatic  transmission).  In  normal 
operation,  the  driver  will  use  only  the  high  range. 

On  the  opposite  side  of  the  gearbox,  on  the  same  shaft,  is  an  electric  clutch  attached  to  a 
dependable  2-hp  electric  golf  cart  motor  made  by  General  Electric  and  used  in  almost  every 
golf  cart  made  in  the  world.  This  electric  motor  will  provide  a  maximum  speed  of  only  8 
kph,  but  allows  for  extremely  quiet  mobility  operations.  With  a  range  of  more  than  4  icm, 
this  will  allow  the  silent  movement  of  the  STV  near  hostile  forces.  This  innovative  dual 
motor  hybrid  design  combines  the  high  power  capability  and  flexibility  of  a  diesel  with  the 
quiet  operation  of  an  electric  motor  in  a  single  system  that  provides  our  STV  system  with 
unique  operational  capability. 

The  output  of  the  gearbox  is  connected  to  a  drive  shaft  that  extends  through  a  watertight 
seal  to  the  outside  of  the  engine  enclosure  to  deliver  power  to  the  front  and  back  wheels  using 
chain  drives.  The  front  wheels  incorporate  Polaris's  “on-demand”  front-wheel  drive  that  only 
activates  when  slippage  is  detected  in  the  rear  wheels.  This  reduces  the  effort  required  by 
the  operator  to  turn  the  steering  wheels,  reduces  front  wheel  wear,  and  allows  for  the 
distribution  of  maximum  horsepower  to  the  rear  wheels  if  needed  for  high  speed  mobility. 
This  capability  can  be  turned  off  by  a  switch  on  the  steering  column  or  remotely. 

Attached  to  the  frame  will  be  six  sealed,  waterproof,  fiberglass  boxes  of  varying  sizes. 
These  boxes  provide  enough  volume  to  allow  the  system  to  fully  float  with  a  rider.  One  of 
the  boxes  will  house  the  engine,  radiator  and  drive  train.  Two  boxes  under  the  engine 
compartment  will  hold  the  diesel  fuel.  Two  other  boxes  will  be  up  front  to  hold  the  remote 
operator  pendant  and.  to  serve  as  additional  storage  area.  The  area  behind  the  engine 
compartment  in  the  center  of  the  vehicle  is  reserved  for  the  MOB/RSTA  module.  Directly 
behind  this  area  is  a  large  cargo  box  that  houses  all  vehicle  electronics,  200  A-hr  of  batteries, 
the  navigation  electronics  if  necessary,  and  the  10  km.fiber-optic/RF  communication  and 
payout  system.  This  last  compartment  has  a  hinged  lid  like  the  trunk  of  a  car  to  provide  easy 
access  for  maintenance  or  rapid  replacement  of  die  fiber-optic  cable. 

Manual  operation  of  the  STV  will  be  identical  to  that  of  the  commercial  6X6.  All 
current  controls  will  stay  the  same  except  the  operation  of  the  gearbox  from  neutral  to  reverse 
to  high  or  low  forward  which  will  be  performed  by  the  pushing  of  a  button  instead  of  using 
the  shift  lever.  This  feature  makes  our  system  “ready  to  drive”  with  no  need  for  the  operator 
to  disconnect  actuators  before  manual  operation. 

2.2  Vehicle  Control  System 


An  interconnect  block  diagram  of  the  platform  control  system  components  can  be  seen 
in  Figure  3.  The  vehicle  control  system  uses  a  VMEbus  for  the  primary  means  of 
communication  between  system-level  boards.  The  VMEbus  standard  was  selected  as  the 
backplane  of  choice  to  reduce  electrical  reliability  problems.  The  STV  will  be  subjected  to  a 
high-shock,  high-vibration  environment,  and  the  DIN-type  connector  used  for  VME  boards  is 
ideal  for  providing 
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a  reliable  electrical  connection.  Additionally,  every  highly  integrated  printed  circuit  board 
selected  utilizes  the  3U  VME  format,  which  is  even  further  resistant  to  shock  and  vibration 
.  due  to  its  much  smaller  surface  area.  Finally,  the  entire  cardcage  is  shock  mounted  inside  its 
sealed  enclosure.  , 

The  processor  board  chosen  is  a  powerful  25MHz  Motorola  68020  processor,  with 
floating  point  expansion  capability  (Motorola  68881).  The  68020  board  is  100%  CMOS 
design  and  dissipates  less  than  6  watts.  It  supports  up  to  8  Mbytes  of  on-board  dual-ported 
DRAM,  which  is  more  than  enough  for  the  STV  application.  There  are  seven  additional 
boards  on  the  VME  backplane,  with  two  spare  slots  available  for  expansion.  In  addition,  the 
•platform  power  supplies  are  mounted  to  cards  and  slid  into  the  cardcage  to  make  component- 
for-component  swaps  simple. 

Each  of  the  additional  boards  are  also  100%  CMOS  designs,  allowing  the  primary 
electronics  enclosure  to  be  a  sealed  unit,  requiring  no  outside  air  for  cooling.  The  total  power 
consumption  of  all  the  boards  within  the  electronics  box  is  less  than  30  watts.  Every  board  ' 
on  the  backplane,  except  one,  is  off-the-shelf,  and  in  stock.  This  eliminates  the  possibility  of 
schedule  delays  for  delivery,  as  well  as  design  and  debug  time. 

The  vehicle  includes  eight  12-Volt,  50  A-hr  sealed  lead  acid  batteries,  providing  200  A- 
hr  at  24  Volts.  The  raw  power  is  available  for  additional  subsystems  to  tap  off  via  two  high 
capacity  power  strips  located  in  each  battery  compartment.  In  both  the  diesel  driving  mode 
and  the  trolling  mode,  the  ot  tpnt  of  the  100  amp  alternator  more  than  compensates  for  the 
current  required  for  the  actuators  and  control  electronics. 

When  the  engines  are  off,  two  basic  modes  are  supported.  In  surveillance  mode,  power 
to  all  the  actuators  on  the  platform  are  turned  off  (under  computer  control)  and  only  the  main 
control  electronics,  and  platform  status  sensors  are  operable.  The  power  draw  is  less  than  2 
amps  at  24  volts,  and  with  full  battery  capacity,  the  platform  will  remain  in  surveillance 
mode  for  over  3  days,  with  enough  reserve  power  to  start  the  engine. 

The  second  mode  of  operation  is  a  standby  mode.  Only  the  watchdog  safety  system  is 
active,  consuming  an  insignificant  amount  of  power  (less  than  0.5  watts).  The  amount  of 
time  the  system  can  remain  in  standby  mode  will  be  driven  by  the  power  required  by  the  GEE 
communications  system.  Using  the  power  requirements  of  the  communication  system,  the 
time  in  stand-by  mode  will  exceed  5  days. 

The  control  system  allows  for  a  very  simple  in-the-field  calibration  routine,  and 
automatically  updates  EEPROM  as  necessary.  The  calibration  procedure  is  invoked  using 
the  calibration  buttons  on  the  membrane  switch  panel.  Once  in  calibration  mode,  the 
platform  prompts  the  operator  to  move  each  actuator  (all  are  backdriveable)  to  a  particular 
position.  Once  in  position,  the  operator  presses  the  confirm  button,  and  the  computer  will 
read  the  appropriate  position  indicator  and  store  its  value  in  EEPROM.  A  checksum  is 
maintained  for  each  platform  so  the  validity  of  the  EEPROM  is  checked  every  time  power  is 
applied.  If  the  checksum  is  not  correct,  calibration  mode  will  be  entered  automatically,  and 
the  operator  will  be  prompted  to  calibrate  all  actuators  before  beginning  operation. 

The  rear  cargo  area  houses  most  of  the  electronics  required  to  control  the  STV,  and 
provides  ample  room  for  the  GFE  communications  system  and  fiber-optic  payout  canister. 

The  electronics  box  contains  the  Vehicle  Control  System  (VCS),  as  well  as  the  power 
supplies  for  all  vehicle  components.  It  is  a  shock-isolated  unit,  and  because  of  the  selection 
of  low  power  CMOS  boards,  requires  no  outside  air  for  cooling. 

The  power  amplifiers,  also  located  in  the  rear  cargo  area,  are  separated  from  the  main 
electronics  to  eliminate  noise  and  heat  problems  within  the  electronics  box,  and  to  greatly 
simplify  all  high-power  wiring. 

Space  has  been  allocated  on  the  right  rear  of  the  platform  cargo  area  for  the  mounting 


of  a  fluxgate  magnetometer,  used  for  navigation.  Isolation  of  this  heading  sensor  from  the 
rest  of  the  platform  electronics  and  actuators  should  greatly  reduce  the  amount  of 
compensation  required  for  the  dead-reckoning  navigation  calculations. 

The  STV  is  programmed  using  the  “C”  programming  language  for  all  software 
development,  using  the  VxWorks  development  system.  This  decision  was  based  on  the 
immediate  availability  of  the  real-time  kernel  for  the  CMOS  microprocessor  board  chosen, 
the  familiarity  of  the  RST  personnel  with  the  development  system,  and  its  low  cost  per  target 
system.  It  allows  for  real-time  data  capture  and  display  on  a  Sun  workstation  as  well  as  both 
symbolic  level  and  source-level  debugging. 

The  vehicle  control  system  is  responsible  for  receiving  commands  and  providing  status 
information  to  the  communication  system,  while  maintaining  stable  control  of  each  element 
of  the  system.  Provisions  have  been  made  for  the  command  data  to  include  a  vehicle  I.D.  for 
multiple  vehicle  control.  This  is  required  to  allow  multiple  platforms  to  operate  on  common 
RF  frequencies.  All  signals  being  used  conform  to  commercial  computer  interface  standards 
(RS422,  TTL  levels,  ±10V  analog  signals,  etc.)  and  will  readily  interface  to  the 
communication  system,  turret  control  system  or  any  additional  future  platform  components. 

The  RST  team  is  using  a  proven  software  design  methodology  —  a  task  decomposition 
approach  originally  developed  at  the  National  Institute  for  Standards  and  Technology  (NIST). 
The  control  system  architecture  for  the  VCS  is  shown  in  Figure  4.  Each  control  block 
(“level”)  can  be  thought  of  as  a  person  in  a  management  chain.  Its  responsibility  is  to  receive 
a  command  from  its  supervisor,  gather  the  appropriate  status  information  relative  to  that 
command,  and  generate  a  command  to  each  of  its  subordinates,  coordinating  their  efforts  as 
necessary. 

As  an  example  of  how  the  system  works,  if  the  current  command  into  the  MOBILITY 
level  is  “Drive_DieseP\  the  level  would  gather  current  operator  command  sensory 
information  (such  as  the  operator  pushing  on  a  joystick),  and  use  that  information  to 
coordinate  the  activity  of  its  subordinates.  In  this  case  it  would  send  a  “SteerJPos”  command 
to  the  STEERING  level  and  a  “Set_Throttle”  command  to  the  DJTHROTTLE  level,  with  the 
appropriate  parameters.  As  the  command  into  the  MOBILITY  level  is  changed,  so  is  its 
response,  i.e.  if  the  “Drive_Diesel”  command  is  changed  to  “TrolLMode”,  the  joystick  input 
would  then  be  used  to  send  a  velocity  vector  to  the  TROLLING  level. 

All  task  decompositions  continue  down  the  diagram  until  commands  can  be  directly 
executed  in  hardware.  For  example  “Fuel_Solenoid_On”  is  simply  a  high-current  output 
from  the  VDOUT-32  that  throws  the  relay  that  applies  power  to  the  fuel  solenoid. 


The  Mob/RSTA  module  has  three  parts  -  the  elevating  mast,  the  turret ,  and  the  control 
electronics. 


Our  elevating  mast  design  is  shown  in  Figure  5.  The  mast  is  a  scissors  type  design, 
similar  to  the  configuration  used  in  the  elevating  mast  on  the  TOV  system.  In  the  stowed 
configuration,  the  mast  is  only  0.46  meters  tall.  The  mast  will  raise  to  3.54  meters,  providing 
a  total  extension  of  3  meters.  In  the  stowed  configuration  the  mast  has  a  footprint  of  0.46 
meters  by  1.15  meters.  Even  with  a  design  factor  of  safety  of  three,  we  were  able  to  get  the 
weight  of  the  mast  entire  mast  down  to  50  kg..  Mounting  the  mast  to  the  platform  is  very 
simple,  requiring  only  4  bolts. 
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3.2  Turret 

Our  sensor  turret  design  is  shown  in  Figure  6.  The  turret  control  electronics  necessary 
to  communicate  to  and  control  all  of  the  sensors  and  motors  are  housed  in  the  tilt  structure 
enclosure.  The  turret  has  mounting  locations  to  simultaneously  hold  four  camera  enclosures, 
an  LTM-86,  and  an  AN/TAS-4.  The  ability  to  quickly  mount  up  to  four  different  camera 
systems  provides  the  user  with  the  capability  to  tailor  the  the  sensor  suite  to  a  particular 
mission.  By  carrying  both  day  and  nights  camera  systems,  the  STV  system  can  be  used  in  24 
hour  day/night  combined  missions. 

The  pan  axis  is  capable  of  turning  ±  270  degrees  (540  total)  and  the  tilt  axis  ±  90 
degrees.  The  pan  and  tilt  axes  are  powered  with  identical  PMI  precision  gear  servo  motor 
with  100:1  Revex  high  precision,  low  backlash  spur  gearbox,  which  provides  a  velocity  range 
of  0.05  to  150  degrees  per  second.  This  rare  earth  DC  servo  has  a  constant  torque  that  will 
provide  the  same  quick,  accurate  response,  irrespective  of  rotational  speed. 

The  motors  will  use  an  incremental  encoder  for  position  and  velocity  control.  They 
also  will  each  have  a  two-turn  servo  potentiometer  to  furnish  the  absolute  position  upon 
system  start-up.  Without  them,  the  system  would  not  know  the  “Home”  or  “Zero”  position. 

The  structure  of  the  turret  is  made  of  a  welded  combination  of  aluminum  plates  and  ex¬ 
trusions.  The  structure  for  the  tilt  axis  is  a  sealed  box  that  houses  all  the  electronics  to  con¬ 
trol  and  power  the  motors  and  sensors.  The  pan  assembly  sits  on  top  of  a  Kadon  turntable 
bearing  which  provides  smooth  rotational  movement  and  supports  the  weight  of  the  entire 
turret.  The  bearing  and  motor  assembly  sits  in  a  watertight  housing  which  is  bolted  directly 
onto  the  top  plate  of  the  elevating  mast.  The  combined  weight  of  the  motors  and  structure, 
two  driving  and  two  targeting  camera  systems,  the  FLIR  and  LTM-86  and  all  electronics  are 
estimated  to  be  around  45  kg. 

3.3  Mob/RSTA  Control  System 


The  Mob/RSTA  Control  System  (RCS)  incorporates  the  same  basic  structure  as  the 
RST  remote  platform.  An  interconnect  block  diagram  of  the  key  control  system  components 
can  be  seen  in  Figure  7. 

Each  printed  circuit  board  selected  is  a  3U  format,  reducing  both  surface  area  and 
weight,  thus  allowing  for  an  extremely  high  tolerance  level  of  shock  and  vibration.  The  spare 
slots  in  the  cardcage  are  available  for  power  supply  mounting  and  future  expansion.  The 
wiring  harnesses  running  to  these  boards  connect  at  the  front  panel  and  run  to  the  cover  plate, 
making  testpoints  easily  accessible. 

The  cardcage  is  located  within  the  turret  inner  gimbal.  Shock  isolation  of  the  3U 
cardcage  is  not  needed  because  these  cards  have  a  much  higher  natural  frequency,  and 
therefore  are  much  better  equipped  to  withstand  the  low-frequency  vibration  expected  on  a 
small  ground  based  vehicle. 

The  power  amplifiers  selected  come  packaged  in  metal  enclosures  which  provide  for 
noise  isolation  and  the  ability  for  them  to  be  mounted  directly  to  the  surface  of  the  inner 
gimbal,  which  provides  an  excellent  heat  sink. 

The  fuse  box  is  a  compact  off-the-shelf  unit  used  mainly  in  boating  and  other  marine 
applications.  It  can  hold  20  fuses  and  12  high  power  relays,  is  water  resistant,  and  is  cost 
effective  because  of  its  commercial  availability. 

The  processor  board  chosen  is  a  PEP  VM20,  a  16-MHz  Motorola  68020  processor, 
with  floating  point  expansion  capability  (Motorola  68881).  The  68020  board  is  100%  CMOS 
design  and  dissipates  less  than  6  watts.  It  supports  up  to  8  Mbytes  of  on-board  dual-ported 
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Figure  6  Mob/RSTA  Turret  Design 


Figure  7  Mob/RSTA  Module  Electronics  Interconnect  Diagram 
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DRAM,  which  is  more  than  enough  given  the  input/output  requirements,  and  the  application. 
There  are  two  serial  ports  on  the  VM20  configurable  for  either  RS232  or  RS422.  Two 
additional  serial  ports  can  be  added  as  a  daughter  module  if  necessary.  These  serial  ports  will 
be  used  to  communicate  with  the  AN/TAS-4B,  and  the  remote  platform,  or  the 
communications  system,  depending  on  the  final  system  configuration. 

The  remainder  of  the  selected  boards  are  also  100%  CMOS  designs,  allowing  the  inner 
gimbal  electronics  enclosure  to  be  a  sealed  unit,  requiring  no  outside  air  for  cooling.  The 
total  power  consumption  of  all  the  boards  within  the  cardcage  is  less  than  24  watts.  Every 
board  on  the  backplane,  except  one,  is  off-the-shelf,  and  stocked.  This  eliminates  the 
possibility  of  schedule  delays  for  delivery,  as  well  as  design  and  debug  time.  The  only  board 
not  available  off-the-shelf  is  the  DIB,  which  was  customized  to  control  the  complex  interface 
of  the  LTM-86  laser  designator  and  rangefinder. 

The  software  design  methodology  for  the  Mob/RSTA  Module  is  identical  to  the 
Remote  Platform.  Referring  back  to  Figure  4,  if  TURRET  receives  the  command 
“Semi_Auto_Track”  and  a  vector  specifying  the  velocity  with  which  to  maintain  the  track,  it 
performs  the  simple  computation  to  isolate  the  pan  and  tilt  velocity  components  of  that 
vector,  then  issues  the  command  “Move_vel”  to  each  of  the  PAN_MOTOR  and 
TELT_MOTOR  levels  with  their  respective  parameters. 

Because  the  methodology  is  extremely  modular,  it  is  easily  extendable  to  include 
additional  levels,  or  more  commands  to  existing  levels.  For  example,  the  addition  of  a  video 
autotrack  would  not  require  any  significant  modification  to  the  present  design.  A 
“Video_Auto_Track”  command  would  simply  be  added  to  the  TURRET  level,  and  the 
parameters  to  it  would  specify  a  position  in  space  where  the  cross-hairs  of  the  camera  should 
be  maintained.  The  TURRET  level's  responsibility  for  the  new  command  would  be  to 
calculate  the  pan  and  tilt  angles  required  to  point  at  the  given  position,  and  issue  the 
subcommand  “Move_pos”  to  both  the  PAN  JMOTOR  and  TELT_MOTOR  levels  with  the 
proper  angle  parameters.  The  repetitive  execution  of  these  control  cycles  will  cause  the 
system  to  track  the  output  of  the  autotracker  over  time. 
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